Display device, method of controlling display device, and program

ABSTRACT

An HMD includes an image display unit that makes a user visually perceive an image, and an control device, capable of being moved independently of the image display unit, which includes a six-axis sensor detecting a movement and a track pad receiving an operation. The HMD controls the display of the image display unit in accordance with the movement detected by the six-axis sensor in a state where the track pad receives an operation.

BACKGROUND

1. Technical Field

The present invention relates to a display device, a method ofcontrolling the display device, and a program.

2. Related Art

Hitherto, there has been known an example in which an operation isperformed on a display device using a device which is used by being heldin a user's hand (for example, see JP-T-2009-500923). A remote controldevice disclosed in JP-T-2009-500923 includes a ring-shaped housing orbody and is operated by being gripped by a user, and the movement of theremote control device is reflected on display.

In a configuration disclosed in JP-T-2009-500923, the operation of theremote control device is reflected on display. In this manner, in aconfiguration in which display is controlled in response to a movement,there is an advantage in that operability is excellent because of asimple operation. However, the display changes in response to amovement, regardless of a user's intention. For this reason, there isthe possibility that a user's unintended change in display occurs.

SUMMARY

An advantage of some aspects of the invention is to provide a displaydevice capable of easily changing display in response to an operationand suppressing a user's unintended change in display, a method ofcontrolling the display device, and a program.

A head-mounted display device according to an aspect of the inventionincludes a display unit that makes a user visually perceive an image,the head-mounted display device including an operation device, capableof being moved independently of the display unit, which includes amovement detection unit detecting a movement of the operation device andan operation reception unit receiving an operation different from thedetection of the movement detection unit, and a control unit thatcontrols display of the display unit in accordance with the movementdetected by the movement detection unit in a state where the operationreception unit receives an operation.

According to the aspect of the invention, it is possible to easilycontrol the display of the display unit in accordance with an operationof moving the operation device. In addition, it is possible to prevent auser's unintended change in display according to the operation state ofthe operation reception unit.

In the display device of the aspect of the invention, the operationreception unit may receive at least any one of a pressing operation anda contact operation.

According to the aspect of the invention with this configuration, it ispossible to move the operation device and to increase operability in acase where the operation reception unit is operated. In addition,operability in a case where the operation reception unit is operatedwithout being visually perceived is excellent, and thus it is possibleto easily and reliably operate the operation reception unit even when auser wearing the head-mounted display device hardly visually perceivethe operation device.

In the display device of the aspect of the invention, the control unitmay change a display magnification of an image displayed on the displayunit in a case where a movement corresponding to an operation ofrotating the operation device is detected by the movement detectionunit.

According to the aspect of the invention with this configuration, it ispossible to change a display magnification in accordance with aninstinctive operation.

In the display device of the aspect of the invention, the control unitmay change an image displayed on the display unit in a direction whichis set in advance, in a case where a movement corresponding to anoperation of lifting down the operation device or an operation oflifting up the operation device is detected by the movement detectionunit.

According to the aspect of the invention with this configuration, it ispossible to change an image displayed on the display unit in accordancewith an instinctive operation.

In the display device of the aspect of the invention, the control unitmay change display of the display unit in a state where the operationreception unit receives an operation, and then fixedly sets a change inthe display of the display unit in a case where the operation iscanceled.

According to the aspect of the invention with this configuration, it ispossible to perform an operation of fixedly setting a change in thedisplay of the display unit. For this reason, it is possible to increasethe degree of freedom of a change in display and to achieve animprovement in operability.

In the display device of the aspect of the invention, the control unitmay have a predetermined function set therein which is performed in acase where the operation reception unit receives an operation, and mayperform a normal mode in which the predetermined function is performedin a case where the operation reception unit receives an operation and adisplay operation mode in which the display of the display unit iscontrolled in accordance with the movement detected by the movementdetection unit in a state where the operation reception unit receives anoperation, in a changeover manner.

According to the aspect of the invention with this configuration, it ispossible to use the operation reception unit as an operation unit thatindicates a predetermined function.

In the display device of the aspect of the invention, the control unitmay perform the display operation mode in a case where the operationreception unit receives an operation within a set time after themovement detection unit detects start of a movement.

According to the aspect of the invention with this configuration, it ispossible to easily select whether to perform the normal mode or toperform the display operation mode in accordance with the operation ofthe operation reception unit.

In the display device of the aspect of the invention, in a case wherethe control unit starts performing the display operation mode, thecontrol unit may reflect a detection history detected by the movementdetection unit during a period between when the movement detection unitdetects start of a movement and when the operation reception unitreceives an operation, to thereby control display of the display unit.

According to the aspect of the invention with this configuration, themovement of the operation device until the start of the displayoperation mode can be reflected on the control of display.

In the display device of the aspect of the invention, the operationdevice may include a position indication operation unit that receives aposition indication operation.

According to the aspect of the invention with this configuration, it ispossible to perform the position indication operation using theoperation device.

In the display device of the aspect of the invention, the control unitmay change display of the display unit on the basis of a displayposition corresponding to a position which is indicated by a positionindication operation received by the position indication operation unitin a state where the position indication operation is received by theposition indication operation unit and the operation reception unitreceives an operation.

According to the aspect of the invention with this configuration, it ispossible to control the display of the display unit with a high level offreedom by using the position indication operation and the movement ofthe operation device. In addition, it is possible to perform complicateddisplay control by a simple operation and to achieve an improvement inoperability.

In the display device of the aspect of the invention, the control unitmay display an image displayed on the display unit in an enlarged orreduced manner, centering on the display position corresponding to theposition which is indicated by the position indication operationreceived by the position indication operation unit.

According to the aspect of the invention with this configuration, it ispossible to realize control of enlarging or reducing display centeringon a user's desired position by a simple operation using the operationdevice.

In the display device of the aspect of the invention, the positionindication operation unit may function as the operation reception unit.

According to the aspect of the invention with this configuration,operability in a case where the operation device is operated withoutbeing visually perceived is excellent because of a small number ofobjects to be operated, and thus it is possible to easily and reliablyoperate the operation device even when a user wearing the head-mounteddisplay device hardly visually perceives the operation device. Inaddition, this is advantageous in a case where the operation device ismade small.

The display device of the invention may further include a display unitmovement detection unit that detects a movement of the display unit,wherein the control unit may specify an indication position of anoperation received by the position indication operation unit at a timingwhen the display unit movement detection unit detects a predeterminedmovement.

According to the aspect of the invention with this configuration, anoperation of fixedly setting an indication position during a positionindication operation can be performed by the movement of the displayunit. For this reason, it is possible to further improve operability ina case where a user wearing the head-mounted display device operates theoperation device.

In the display device of the aspect of the invention, the control unitmay be capable of performing a first control of controlling display ofthe display unit in accordance with a type of operation corresponding toa movement detected by the movement detection unit in a state where theoperation reception unit receives an operation, a second control ofchanging display of the display unit on the basis of a display positioncorresponding to a position which is indicated by a position indicationoperation received by the position indication operation unit, and athird control of changing display of the display unit in response to adirection of the movement detected by the movement detection unit, in achangeover manner.

According to the aspect of the invention with this configuration, it ispossible to change the display of the display unit in accordance withthree controls, in response to the movement of the operation device.

In the display device of the aspect of the invention, the control unitmay perform control of moving an image displayed on the display unit ina first display operation direction and a second display operationdirection paired with the first display operation direction. The firstdisplay operation direction may be associated with a first movementdirection detected by the movement detection unit, and the seconddisplay operation direction may be associated with a second movementdirection detected by the movement detection unit.

According to the aspect of the invention with this configuration, it ispossible to move an image displayed on the display unit by moving theoperation device and to indicate a direction of movement of the image inaccordance with a movement direction of the operation device.

In the display device of the aspect of the invention, the control unitmay start moving an image in the first display operation direction inresponse to a movement in the first movement direction detected by themovement detection unit, and then may stop moving the image in the firstdisplay operation direction in a case where a movement in the secondmovement direction is detected by the movement detection unit.

According to the aspect of the invention with this configuration, it ispossible to move the image displayed on the display unit to a desiredposition by combining the movements of the operation device in the twodirections.

In the display device of the aspect of the invention, the control unitmay start moving an image in the first display operation direction orthe second display operation direction in response to a movementdetected by the movement detection unit, and then may stop moving theimage displayed on the display unit in a case where the operationreceived by the operation reception unit is canceled.

According to the aspect of the invention with this configuration, it ispossible to move the image displayed on the display unit to a desiredposition by combining the movement of the operation device and anoperation of the operation reception unit.

In the display device of the aspect of the invention, the operationdevice may be a wearable device which is worn on the user's body.

According to the aspect of the invention with this configuration, it ispossible to easily control the display of the display unit in accordancewith an operation of moving a user's body. In addition, a user'sunintended change in display can be prevented by the operation state ofthe operation reception unit, and thus it is possible to moreinstinctively perform an operation of controlling display.

A head-mounted display device according to another aspect of theinvention includes a display unit that makes a user visually perceive animage, the head-mounted display device including an operation device,capable of being moved independently of the display unit, which includesa movement detection unit detecting a movement of the operation device,and a control unit that controls display of the display unit inaccordance with the movement detected by the movement detection unit,wherein the control unit performs a first display change correspondingto a movement in a first movement direction which is detected by themovement detection unit and a second display change corresponding to amovement in a second movement direction which is detected by themovement detection unit, as a process performed on an image displayed onthe display unit, and wherein the control unit starts the first displaychange in a case where the movement in the first movement direction isdetected, and then stops the first display change in a case where themovement in the second movement direction is detected by the movementdetection unit.

According to the aspect of the invention, it is possible to easilycontrol the display of the display unit in accordance with an operationof moving the operation device. In addition, it is possible to move theimage displayed on the display unit to a desired position by combiningthe movements of the operation device in the two directions.

In the display device of the aspect of the invention, the control unitmay perform the first display change in a case where the movement in thefirst movement direction is detected by the movement detection unit, andmay perform the second display change in a case where the movement inthe second movement direction is detected by the movement detectionunit.

According to the aspect of the invention with this configuration, it ispossible to move an image displayed on the display unit by moving theoperation device and to indicate a direction of movement of the image inaccordance with a movement direction of the operation device.

In the display device of the aspect of the invention, the operationdevice may include an operation reception unit that receives anoperation, and the control unit may change display of the display unitin accordance with the movement detected by the movement detection unitin a state where the operation reception unit receives an operation.

According to the aspect of the invention with this configuration, it ispossible to prevent a user's unintended change in display according tothe operation state of the operation reception unit.

In the display device of the aspect of the invention, the control unitmay perform the first display change in a case where the movement in thefirst movement direction is detected by the movement detection unit in astate where the operation reception unit receives an operation.

According to the aspect of the invention with this configuration, it ispossible to prevent a user's unintended change in display according tothe operation state of the operation reception unit.

In the display device of the aspect of the invention, the control unitmay start the first display change in a case where the movement in thefirst movement direction is detected by the movement detection unit, andthen may stop the first display change in a case where the operationreception unit does not receive an operation.

According to the aspect of the invention with this configuration, it ispossible to prevent a user's unintended change in display according tothe operation state of the operation reception unit.

In the display device of the aspect of the invention, the control unitmay perform the second display change in a case where the movement inthe second movement direction is detected by the movement detection unitin a state where the operation reception unit receives an operation.

According to the aspect of the invention with this configuration, it ispossible to move an image displayed on the display unit by moving theoperation device and to indicate a direction of movement of the image inaccordance with a movement direction of the operation device.

In the display device of the aspect of the invention, the operationreception unit may receive an operation different from the detection ofthe movement detection unit.

According to the aspect of the invention with this configuration, it ispossible to select a state in which display is controlled by themovement of the operation device and a state where display is notcontrolled, in accordance with an operation which is different from themovement detected by the movement detection unit, and thus it ispossible to more reliably prevent a user's unintended change in display.

In the display device of the aspect of the invention, the operationdevice may be a wearable device which is worn on the user's body.

According to the aspect of the invention with this configuration, it ispossible to easily control the display of the display unit in accordancewith an operation of moving a user's body. In addition, a user'sunintended change in display can be prevented by the operation state ofthe operation reception unit, and thus it is possible to moreinstinctively perform an operation of controlling display.

A method of controlling a display device according to still anotheraspect of the invention is a method of controlling a display deviceincluding a display unit that makes a user visually perceive an imageand an operation device, capable of being moved independently of thedisplay unit, which includes a movement detection unit detecting amovement of the operation device and an operation reception unitreceiving an operation, the method including controlling display of thedisplay unit in accordance with the movement detected by the movementdetection unit in a state where the operation reception unit receives anoperation.

According to the aspect of the invention, it is possible to easilycontrol the display of the display unit in accordance with an operationof moving the operation device. In addition, it is possible to prevent auser's unintended change in display according to the operation state ofthe operation reception unit.

A method of controlling a display device according to yet another aspectof the invention is a method of controlling a display device including adisplay unit that makes a user visually perceive an image and anoperation device, capable of being moved independently of the displayunit, which includes a movement detection unit detecting a movement ofthe operation device, the method including performing a first displaychange corresponding to a movement in a first movement direction whichis detected by the movement detection unit and a second display changecorresponding to a movement in a second movement direction which isdetected by the movement detection unit, as a process performed on animage displayed on the display unit, and starting the first displaychange in a case where the movement in the first movement direction isdetected, and then stopping the first display change in a case where themovement in the second movement direction is detected by the movementdetection unit.

According to the aspect of the invention, it is possible to easilycontrol the display of the display unit in accordance with an operationof moving the operation device. In addition, it is possible to move theimage displayed on the display unit to a desired position by combiningthe movements of the operation device in the two directions.

A program according to still yet another aspect of the invention isexecutable by a computer that controls a display device including adisplay unit that makes a user visually perceive an image and anoperation device, capable of being moved independently of the displayunit, which includes a movement detection unit detecting a movement ofthe operation device and an operation reception unit receiving anoperation, and the program causes the computer to perform a function ofcontrolling display of the display unit in accordance with the movementdetected by the movement detection unit in a state where the operationreception unit receives an operation.

According to the aspect of the invention, it is possible to easilycontrol the display of the display unit in accordance with an operationof moving the operation device. In addition, it is possible to prevent auser's unintended change in display according to the operation state ofthe operation reception unit.

A program according to further another aspect of the invention isexecutable by a computer that controls a display device including adisplay unit that makes a user visually perceive an image and anoperation device, capable of being moved independently of the displayunit, which includes a movement detection unit detecting a movement ofthe operation device, and the program causes the computer to performfunctions of performing a first display change corresponding to amovement in a first movement direction which is detected by the movementdetection unit and a second display change corresponding to a movementin a second movement direction which is detected by the movementdetection unit, as a process performed on an image displayed on thedisplay unit, and starting the first display change in a case where themovement in the first movement direction is detected, and then stoppingthe first display change in a case where the movement in the secondmovement direction is detected by the movement detection unit.

According to the aspect of the invention, it is possible to easilycontrol the display of the display unit in accordance with an operationof moving the operation device. In addition, it is possible to move theimage displayed on the display unit to a desired position by combiningthe movements of the operation device in the two directions.

The invention can also be implemented in various aspects other than thedisplay device, the method of controlling a display device, and theprogram which are described above. For example, the invention can beimplemented in an aspect such as a recording medium having theabove-mentioned program recorded thereon, a server device thatdistributes a program, a transmission medium that transmits the program,or a data signal that realizes the program within a carrier wave.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be described with reference to the accompanyingdrawings, wherein like numbers reference like elements.

FIG. 1 is a diagram showing an exterior configuration of an HMD.

FIG. 2 is a diagram showing a configuration of an optical system of animage display unit.

FIGS. 3A and 3B are diagrams showing association between the imagedisplay unit and an imaging range.

FIG. 4 is a block diagram of units constituting the HMD.

FIG. 5 is a block diagram showing a control unit and a storage unit.

FIGS. 6A and 6B are diagrams showing a state of lighting of an LEDdisplay unit.

FIGS. 7A and 7B are diagrams showing an operation example using acontrol device.

FIGS. 8A to 8D are diagrams showing changes in display corresponding tothe operation of the HMD.

FIGS. 9A to 9C are diagrams showing changes in display corresponding tothe operation of the HMD.

FIGS. 10A to 10C are diagrams showing changes in display correspondingto the operation of the HMD.

FIG. 11 is a flow chart showing the operation of the HMD.

FIG. 12 is a flow chart showing the operation of the HMD.

FIG. 13 is a diagram showing the exterior of a timepiece type device.

FIG. 14 is a diagram showing the exterior of a ring type device.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

FIG. 1 is a diagram showing an exterior configuration of a head-mounteddisplay (HMD) 100 according to an embodiment to which the invention isapplied.

The HMD 100 is a display device including an image display unit 20(display unit) that makes a user visually perceive a virtual image in astate of being worn on the user's head, and a control device 10 thatcontrols the image display unit 20. The control device 10 (operationdevice) includes a flat box-shaped case 10A (also referred to as ahousing or a main body) as shown in FIG. 1, and includes units to bedescribed later in the case 10A. Various buttons 11, a switch, a trackpad 14, and the like, which receive a user's operation, are provided onthe surface of the case 10A. The user operates these, and thus thecontrol device 10 functions as a controller of the HMD 100.

The image display unit 20 is a body which is worn on a user's head, andhas a shape of spectacles in the present embodiment. The image displayunit 20 includes a right display unit 22, a left display unit 24, aright light guiding plate 26, and a left light guiding plate 28 in amain body including a right holding portion 21, a left holding portion23, and a front frame 27.

Each of the right holding portion 21 and the left holding portion 23extend backward from both ends of the front frame 27, and holds theimage display unit 20 on a user's head like a temple (bow) ofspectacles. Here, among both ends of the front frame 27, an endpositioned on the right side of the user in a wearing state of the imagedisplay unit 20 is set to be an end ER, and an end positioned on theleft side of the user is set to be an end EL. The right holding portion21 is provided so as to extend from the end ER of the front frame 27 toa position corresponding to the user's right head side in a wearingstate of the image display unit 20. The left holding portion 23 isprovided so as to extend from the end EL to a position corresponding tothe user's left head side in a wearing state of the image display unit20.

The right light guiding plate 26 and the left light guiding plate 28 areprovided at the front frame 27. The right light guiding plate 26 ispositioned in front of the user's right eye in a wearing state of theimage display unit 20, and makes the right eye to visually perceive animage. The left light guiding plate 28 is positioned in front of theuser's left eye in a wearing state of the image display unit 20, andmakes the left eye to visually perceive an image.

The front frame 27 has a shape in which an end of the right lightguiding plate 26 and an end of the left light guiding plate 28 areconnected to each other, and the connection position corresponds to theuser's glabella in a state where the user wears the image display unit20. The front frame 27 may be provided with a nose pad portion abuttingon the user's nose in a wearing state of the image display unit 20, at aposition where the right light guiding plate 26 and the left lightguiding plate 28 are connected to each other. In this case, the imagedisplay unit 20 can be held by the user's head by the nose pad portion,the right holding portion 21, and the left holding portion 23. Inaddition, a belt (not shown) coming into contact with the back of theuser's head in a wearing state of the image display unit 20 may beconnected to the right holding portion 21 and the left holding portion23. In this case, the image display unit 20 can be held by the user'shead by using the belt.

The right display unit 22, which is a unit related to the display of animage using the right light guiding plate 26, is provided at the rightholding portion 21 and is positioned in the vicinity of the user's righthead side in a wearing state. The left display unit 24, which is a unitrelated to the display of an image using the left light guiding plate28, is provided at the left holding portion 23 and is positioned in thevicinity of the user's left head side in a wearing state. Meanwhile, theright display unit 22 and the left display unit 24 are also collectivelyreferred to as a “display driving unit”.

The right light guiding plate 26 and the left light guiding plate 28 ofthe present embodiment are optical units, such as prisms, which areformed of a light transmissive resin or the like, and guide image lightoutput from the right display unit 22 and the left display unit 24 tothe user's eyes.

In addition, a light control plate (not shown) may be provided on thesurface of each of the right light guiding plate 26 and the left lightguiding plate 28. The light control plate is an optical element, havinga thin film shape, which has transmittance varying depending on awavelength region of light, and functions as a so-called wavelengthfilter. For example, the light control plate is disposed so as to coverthe front side of the front frame 27 which is a side opposite to a sideof the user's eyes. Optical characteristics of the light control plateare appropriately selected, and thus it is possible to adjust thetransmittance of light having any wavelength region, such as visiblelight, infrared light, or ultraviolet light, and to adjust the amount ofexternal light which is incident on the right light guiding plate 26 andthe left light guiding plate 28 from the outside and passes through theright light guiding plate 26 and the left light guiding plate 28.

The image display unit 20 guides image light beams respectivelygenerated by the right display unit 22 and the left display unit 24 tothe right light guiding plate 26 and the left light guiding plate 28 andmakes a user to visually perceive a virtual image by the image lightbeams, thereby displaying an image. In a case where external lighthaving passed through the right light guiding plate 26 and left lightguiding plate 28 is incident on a user's eyes from the front of theuser, image light and external light constituting a virtual image areincident on the user's eyes, and the visibility of the virtual image isaffected by the intensity of the external light. For this reason, forexample, a light control plate is mounted on the front frame 27, andoptical characteristics of the light control plate are appropriatelyselected or adjusted, thereby allowing the easiness of visual perceptionof the virtual image to be adjusted. In a typical example, it ispossible to use a light control plate having a light transmittance suchan extent that a user wearing the HMD 100 can visually perceive at leastoutside scene. In addition, when the light control plate is used, it ispossible to expect effects of protecting the right light guiding plate26 and the left light guiding plate 28 and suppressing the damage of theright light guiding plate 26 and the left light guiding plate 28, theattachment of dirt thereto, and the like. The light control plate may beconfigured to be attached and detached to and from the front frame 27 oreach of the right light guiding plate 26 and the left light guidingplate 28, a plurality of types of light control plates may beexchangeably mounted, or the light control plate may be omitted.

A camera 61 is disposed at the front frame 27 of the image display unit20. It is desired that the camera 61 images a direction of an outsidescene which is visually perceived by a user wearing the image displayunit 20, and the camera is provided at a position that does not shieldexternal light passing through the right light guiding plate 26 and theleft light guiding plate 28 on the front surface of the front frame 27.In the example of FIG. 1, the camera 61 is disposed on the end ER sideof the front frame 27. The camera 61 may be disposed on the end EL side,or may be disposed at a connecting portion between the right lightguiding plate 26 and the left light guiding plate 28.

The camera 61 is a digital camera including an imaging element, such asa CCD or a CMOS, an imaging lens, and the like. The camera 61 of thepresent embodiment is a monocular camera, but may be constituted by astereo camera. The camera 61 images at least a portion of an outsidescene (real space) in the front direction of the HMD 100, in otherwords, a direction of a user's field of vision in a wearing state of theHMD 100. In another expression, it can be said that the camera 61performs imaging in a range or a direction which overlaps the user'sfield of vision and performs imaging in the user's eye gaze direction.The range of an angle of view of the camera 61 can be appropriately set,but includes an outside world which is visually perceived by the userthrough the right light guiding plate 26 and the left light guidingplate 28 in the present embodiment, as described later. More preferably,an imaging range of the camera 61 is set so as to be capable of imagingthe entire field of vision of the user which is capable of beingvisually perceived through the right light guiding plate 26 and the leftlight guiding plate 28.

The camera 61 performs imaging under the control of an imaging controlunit 149 included in a control unit 150 (FIG. 5), and outputs capturedimage data to the imaging control unit 149.

The HMD 100 may include a distance sensor (not shown) that detects adistance to an object to be measured which is positioned in a presetmeasurement direction. For example, the distance sensor can be disposedat a connecting portion between the right light guiding plate 26 and theleft light guiding plate 28 in the front frame 27. In this case, theposition of the distance sensor is substantially the middle between auser's both eyes in a horizontal direction in a wearing state of theimage display unit 20, and is above the user's both eyes in a verticaldirection. The measurement direction of the distance sensor can be setto be, for example, the front direction of the front frame 27, in otherwords, a direction overlapping an imaging direction of the camera 61.The distance sensor can be configured to include a light source, such asan LED or a laser diode, and a light reception portion that receivesreflected light of light, emitted from the light source, which isreflected from the object to be measured. The distance sensor mayperform a triangulation process or a distance measurement process basedon a time difference under the control of the control unit 150. Inaddition, the distance sensor may be configured to include a soundsource that generates ultrasonic waves and a detection unit thatreceives ultrasonic waves reflected from an object to be measured. Inthis case, the distance sensor may perform a distance measurementprocess on the basis of a time difference to the reflection ofultrasonic waves under the control of the control unit 150.

FIG. 2 is a plan view showing main components in a configuration of anoptical system included in the image display unit 20. FIG. 2 shows aleft eye LE and a right eye RE of a user for convenience of description.

As shown in FIG. 2, the right display unit 22 and the left display unit24 are configured to be bilaterally symmetrical to each other. As aconfiguration in which the user's right eye RE visually perceives animage, the right display unit 22 includes an organic light emittingdiode (OLED) unit 221 that emits image light, and a right optical system251 that includes a lens group guiding image light L emitted by the OLEDunit 221, and the like. The image light L is guided to the right lightguiding plate 26 by the right optical system 251.

The OLED unit 221 includes an OLED panel 223, and an OLED drivingcircuit 225 that drives the OLED panel 223. The OLED panel 223 is aself-emitting display panel which is configured such that light emittingelements respectively emitting red (R), green (G), and blue (B) lightbeams by organic electroluminescence are arranged in a matrix. The OLEDpanel 223 includes a plurality of pixels with a unit having R, G, and Belements one by one as one pixel, and forms an image by pixels that arearranged in a matrix. The OLED driving circuit 225 performs selection ofthe light emitting element included in the OLED panel 223 and electricalconduction to the light emitting element under the control of thecontrol unit 150 (FIG. 5), thereby making the light emitting element ofthe OLED panel 223 emit light. The OLED driving circuit 225 is fixed tothe rear surface of the OLED panel 223, that is, the back of a lightemitting surface by bonding or the like. The OLED driving circuit 225 isconstituted by, for example, a semiconductor device that drives the OLEDpanel 223, and may be mounted on a substrate (not shown) which is fixedto the rear surface of the OLED panel 223. A temperature sensor 217 ismounted on the substrate.

Meanwhile, the OLED panel 223 may be configured such that light emittingelements emitting white light beams are arranged in a matrix and colorfilters corresponding to R, G, and B colors are arranged so as tooverlap each other. In addition, a WRGB OLED panel 223 including a lightemitting element emitting a white (W) light beam, in addition to lightemitting elements emitting R, G, and B light beams, may be used.

The right optical system 251 includes a collimate lens that collimatesimage light L emitted from the OLED panel 223 into a parallel luminousflux. The image light L collimated into a parallel luminous flux by thecollimate lens is incident on the right light guiding plate 26. Aplurality of reflection surfaces reflecting the image light L are formedin a light path that guides light inside the right light guiding plate26. The image light L is guided to the right eye RE side through aplurality of times of reflection inside the right light guiding plate26. A half mirror 261 (reflection surface) which is positioned in frontof the right eye RE is formed in the right light guiding plate 26. Theimage light L is reflected from the half mirror 261 and is emitted fromthe right light guiding plate 26 toward the right eye RE, and the imagelight L forms an image on the retina of the right eye RE, therebyallowing a user to visually perceive the image.

In addition, as a configuration in which an image is visually perceivedby a user's left eye LE, the left display unit 24 includes an OLED unit241 that emits image light, and a left optical system 252 that includesa lens group guiding image light L emitted by the OLED unit 241, and thelike. The image light L is guided to the left light guiding plate 28 bythe left optical system 252.

The OLED unit 241 includes an OLED panel 243, and an OLED drivingcircuit 245 that drives the OLED panel 243. The OLED panel 243 is aself-emitting display panel which is configured in the same manner asthe OLED panel 223. The OLED driving circuit 245 performs selection of alight emitting element included in the OLED panel 243 and electricalconduction to the light emitting element under the control of thecontrol unit 150 (FIG. 5), thereby making the light emitting element ofthe OLED panel 243 emit light. The OLED driving circuit 245 is fixed tothe rear surface of the OLED panel 243, that is, the back of a lightemitting surface by bonding or the like. The OLED driving circuit 245 isconstituted by, for example, a semiconductor device that drives the OLEDpanel 243, and may be mounted on a substrate (not shown) which is fixedto the rear surface of the OLED panel 243. A temperature sensor 239 ismounted on the substrate.

The left optical system 252 includes a collimate lens that collimatesimage light L emitted from the OLED panel 243 into a parallel luminousflux. The image light L collimated into a parallel luminous flux by thecollimate lens is incident on the left light guiding plate 28. The leftlight guiding plate 28 is an optical element in which a plurality ofreflection surfaces reflecting the image light L are formed, and is, forexample, a prism. The image light L is guided to the left eye LE sidethrough a plurality of times of reflection inside the left light guidingplate 28. A half mirror 281 (reflection surface) which is positioned infront of the left eye LE is formed in the left light guiding plate 28.The image light L is reflected from the half mirror 281 and is emittedfrom the left light guiding plate 28 toward the left eye LE, and theimage light L forms an image on the retina of the left eye LE, therebyallowing a user to visually perceive the image.

According to this configuration, the HMD 100 functions as a see-throughtype display device. That is, the image light L reflected from the halfmirror 261 and external light OL having passed through the right lightguiding plate 26 are incident on the user's right eye RE. In addition,the image light L reflected from the half mirror 281 and external lightOL having passed through the half mirror 281 are incident on the lefteye LE. In this manner, the HMD 100 makes image light L of an imageprocessed therein and external light OL, overlapping each other,incident on a user's eyes. The user can view an outside scene throughthe right light guiding plate 26 and the left light guiding plate 28,and the image based on the image light L is visually perceived so as tooverlap the outside scene.

The half mirrors 261 and 281 are image extraction units that reflectimage light beams respectively output by the right display unit 22 andthe left display unit 24 to extract an image, and can be referred to asdisplay units.

Meanwhile, the left optical system 252 and the left light guiding plate28 are also collectively referred to as a “left light guiding portion”,and the right optical system 251 and the right light guiding plate 26are collectively referred to as a “right light guiding portion”.Configurations of the right light guiding portion and the left lightguiding portion are not limited to the above-mentioned example, and anysystem can be used as long as a virtual image is formed in front of auser's eyes using image light. For example, a diffraction grating may beused, or a semi-transmissive reflection film may be used.

Referring back to FIG. 1, the control device 10 and the image displayunit 20 are connected to each other by a connection cable 40. Theconnection cable 40 is detachably connected to a connector (not shown)which is provided below the case 10A, and is connected to variouscircuits provided inside the image display unit 20 from the tip end ofthe left holding portion 23. The connection cable 40 may include a metalcable or an optical fiber cable that transmits digital data, or mayinclude a metal cable that transmits an analog signal. A connector 46 isprovided in the middle of the connection cable 40. The connector 46 is ajack for connecting a stereo mini plug, and the connector 46 and thecontrol device 10 are connected to each other, for example, through aline that transmits an analog sound signal. In a configuration exampleshown in FIG. 1, a headset 30 including right and left ear phones 32 and34 constituting a stereo headphone and a microphone 63 is connected tothe connector 46.

The microphone 63, which is disposed such that a sound collectionportion of the microphone 63 is directed in a user's eye gaze direction,for example, as shown in FIG. 1, collects a sound and outputs a soundsignal to a sound interface 182 (FIG. 4). The microphone 63 may be, forexample, a monaural microphone or a stereo microphone, may be amicrophone having directivity, or may be a non-directional microphone.

The control device 10 includes buttons 11, an LED indicator 12, a trackpad 14, up and down keys 15, a changeover switch 16, and a power supplyswitch 18 as units to be operated by a user. These units to be operatedare disposed on the surface of the case 10A.

The buttons 11 include a menu key for performing the operation of theoperating system 143 (FIG. 5) which is performed by the control device10, a home key, a return key, and the like. In particular, the keys andthe switches include keys and switches which are displaced by a pressingoperation. The LED indicator 12 is turned on or turned off in responseto an operation state of the HMD 100. The up and down keys 15 are usedto input an instruction for turning up or down sound volume which isoutput from the right ear phone 32 and the left ear phone 34 and toinput an instruction for increasing or decreasing the brightness ofdisplay of the image display unit 20. The changeover switch 16 is aswitch that changes over an input corresponding to the operation of theup and down keys 15. The power supply switch 18 is a switch that changesover the turn-on and turn-off of the power supply of the HMD 100, and isconstituted by, for example, a slide switch.

The track pad 14 (operation reception unit, position indicationoperation unit) has an operation surface for detecting a contactoperation, and outputs an operation signal in response to an operationperformed on the operation surface. A detection method in the operationsurface is not limited, and an electrostatic method, a pressuredetection method, an optical method, or the like can be adopted. Contact(touch operation) with the track pad 14 is detected by the touch sensor13 (FIG. 4) to be described later.

In addition, as indicated by a dashed line in FIG. 1, the track pad 14is provided with an LED display unit 17. The LED display unit 17includes a plurality of LEDs, and an operation unit can be visuallyperceived on the LED display unit 17 by the lighting of the LEDs. In theexample of FIG. 1, three marks of Δ (triangle), ∘ (circle), and □(quadrangle) are shown when the LEDs of the LED display unit 17 areturned on. In a state where the LED display unit 17 is turned off, thesemarks cannot be visually perceived. This configuration can be realizedby constituting the track pad 14, for example, by a colored or colorlesstransparent flat plate having a light-transmissive property anddisposing an LED just below the flat plate.

The LED display unit 17 functions as a software button. For example,while the LED display unit 17 is being turned on, a lighting position(display position) of a mark functions as a button for performing anindication corresponding to the mark. In the example of FIG. 1, the mark∘ (circle) functions as a home button, and the control unit 150 to bedescribed later detects the operation of the home button on the basis ofa detected value of the touch sensor 13 when an operation of contactingthe position of the mark ∘ (circle) is performed. In addition, the mark□ (quadrangle) functions as a history button. When an operation ofcontacting the position of the mark □ (quadrangle) is performed, thecontrol unit 150 to be described later detects the operation of thehistory button on the basis of a detected value of the touch sensor 13.In addition, the mark Δ (triangle) functions as a return button. When anoperation of contacting the position of the mark Δ (triangle) isperformed, the control unit 150 to be described later detects theoperation of the return button on the basis of a detected value of thetouch sensor 13.

FIGS. 3A and 3B are diagrams showing main components of the imagedisplay unit 20. FIG. 3A is a perspective view showing main componentswhen the image display unit 20 is seen from a user's head side, and FIG.3B is a diagram showing an angle of view of the camera 61. Meanwhile, inFIG. 3A, the connection cable 40 is not shown.

FIG. 3A shows a side coming into contact with the head of a user of theimage display unit 20, in other words, a side viewed by the user's righteye RE and left eye LE. In other words, the back sides of the rightlight guiding plate 26 and the left light guiding plate 28 are viewed.

In FIG. 3A, the half mirror 261 irradiating the user's right eye RE withimage light and the half mirror 281 irradiating the user's left eye LEwith image light are viewed as a substantially quadrangular region. Inaddition, the entirety of the right light guiding plate 26 and the leftlight guiding plate 28 respectively including the half mirrors 261 and281 transmit external light as described above. For this reason, anoutside scene is visually perceived by the user through the entirety ofthe right light guiding plate 26 and the left light guiding plate 28,and a rectangular display image is visually perceived at the positionsof the half mirrors 261 and 281.

The camera 61 is disposed at the right end in the image display unit 20as described above, and performs imaging in a direction in which botheyes of a user are directed, that is, images a front side for the user.FIG. 3B is a schematic diagram showing the position of the camera 61together with a right eye RE and a left eye LE of a user, when seen in aplan view. An angle of view (imaging range) of the camera 61 is denotedby C. Meanwhile, FIG. 3B shows the angle of view C in the horizontaldirection, but a real angle of view of the camera 61 is also extended inthe vertical direction, similar to a general digital camera.

An optical axis of the camera 61 is set to be a direction including eyegaze directions of the right eye RE and the left eye LE. An outsidescene capable of being visually perceived by a user wearing the HMD 100is not limited to infinity. For example, as shown in FIG. 3B, when theuser's both eyes gaze at an object OB, the user's eye gaze is directedto the object OB as indicated by signs RD and LD in the drawing. In thiscase, a distance from the user to the object OB is in a range ofapproximately 30 cm to approximately 10 m in many cases, and is in arange of approximately 1 m to approximately 4 m in more cases.Consequently, standards of an upper limit and a lower limit of adistance from the user to the object OB during the normal use may bedetermined with respect to the HMD 100. The standards may be obtained byexamination or experiment, or may be set by a user. It is preferablethat the optical axis and angle of view of the camera 61 are set so thatthe object OB is included in the angle of view in a case where adistance to the object OB during the normal use is equivalent to the setstandard of the upper limit and is equivalent to the set standard of thelower limit.

In addition, generally, a human's viewing angle is set to approximately200 degrees in the horizontal direction and is set to approximately 125degrees in the vertical direction. Here, an effective field of viewexcellent in an information reception capacity is approximately 30degrees in the horizontal direction and is approximately 20 degrees inthe vertical direction. Further, a stable eye gaze field in which ahuman's eye gaze point is rapidly and stably viewed is set to 60 degreesto 90 degrees in the horizontal direction and is set to 45 degrees to 70degrees in the vertical direction. In this case, when the eye gaze pointis the object OB of FIG. 3B, 30 degrees in the horizontal direction and20 degrees in the vertical direction centering on eye gazes RD and LDare effective fields of view. In addition, approximately 60 degrees to90 degrees in the horizontal direction and approximately 45 degrees to70 degrees in the vertical direction are stable eye gaze fields, andapproximately 200 degrees in the horizontal direction and approximately125 degrees in the vertical direction are viewing angles. Further, areal field of view visually perceived by a user through the imagedisplay unit 20 and the right and left light guiding plates 26 and 28can be referred to as a real field of view (FOV). In the configurationof the present embodiment shown in FIGS. 1 and 2, a real field of viewis equivalent to a real field of view which is visually perceived by auser through the right light guiding plate 26 and the left light guidingplate 28. The real field of view is narrower than a viewing angle and astable eye gaze field, but is wider than an effective field of view.

It is preferable that the angle of view C of the camera 61 allows arange wider than a user's field of view to be capable of being imaged.Specifically, it is preferable that the angle of view C is wider than atleast a user's effective field of view. In addition, it is morepreferable that the angle of view C is wider than a user's real field ofview. More preferably, the angle of view C is wider than a user's stableeye gaze field, and most preferably, the angle of view C is wider than aviewing angle of a user's both eyes.

The camera 61 includes a so-called wide angle lens as an imaging lens,and may be configured to be capable of performing imaging at a wideangle of view. The wide angle lens may include lenses which are called asuper-wide angle lens and a semi-wide angle lens, or may be a singlefocus lens or a zoom lens. Alternatively, the camera 61 may beconfigured to include a lens group constituted by a plurality of lenses.

FIG. 4 is a block diagram showing the configuration of unitsconstituting the HMD 100.

The control device 10 includes a main processor 140 that executes aprogram to control the HMD 100. A memory 118 and a non-volatile storageunit 121 are connected to the main processor 140. In addition, the trackpad 14 and an operation unit 110 as input devices are connected to themain processor 140. In addition, a six-axis sensor 111, a magneticsensor 113, and a GPS 115, as sensors, are connected to the mainprocessor 140. In addition, a communication unit 117, a sound codec 180,an external connector 184, an external memory interface 186, a USBconnector 188, a sensor hub 192, and an FPGA 194 are connected to themain processor 140. These function as interfaces for connection to theoutside.

The main processor 140 is mounted on a controller substrate 120 builtinto the control device 10. In addition to the main processor 140, thememory 118, the non-volatile storage unit 121, and the like may bemounted on the controller substrate 120. In the present embodiment, thesix-axis sensor 111, the magnetic sensor 113, the GPS 115, thecommunication unit 117, the memory 118, the non-volatile storage unit121, the sound codec 180, and the like are mounted on the controllersubstrate 120. In addition, a configuration may also be adopted in whichthe external connector 184, the external memory interface 186, the USBconnector 188, the sensor hub 192, the FPGA 194, and the interface 196are mounted on the controller substrate 120.

In a case where the main processor 140 executes a program, the memory118 constitutes a work area in which the program to be executed and datato be processed are temporarily stored. The non-volatile storage unit121 is constituted by a flash memory and an embedded multimedia card(eMMC). The non-volatile storage unit 121 stores programs executed bythe main processor 140, and various pieces of data which are processedby the execution of the main processor 140.

The main processor 140 detects a contact operation with respect to theoperation surface of the track pad 14 on the basis of an operationsignal which is input from the track pad 14, and acquires the operationposition thereof.

The operation unit 110 includes the buttons 11, the touch sensor 13, andthe LED display unit 17. The touch sensor 13 detects a touch operationperformed on the track pad 14, and specifies the operation position ofthe detected touch operation. In a case where the operation of thebuttons 11 is performed and the touch sensor 13 detects a touchoperation, an operation signal is output from the operation unit 110 tothe main processor 140.

The LED display unit 17 includes an LED (not shown) which is disposedjust below the track pad 14 (FIG. 1), and a driving circuit for turningon the LED. The LED display unit 17 turns on, flickers, and turns offthe LED under the control of the main processor 140.

Light emission patterns of the LED of the LED display unit 17 are shownin FIGS. 6A and 6B.

FIGS. 6A and 6B are timing charts showing changes in the luminance ofthe LED of the LED display unit 17. In the light emission pattern shownin FIG. 6A, the LED display unit 17 periodically turns on the LED, andthe luminance of the LED sharply rises and falls. On the other hand, inthe light emission pattern of FIG. 6B, the luminance of the LED changesslowly. The LED display unit 17 turns on the LED with the pattern shownin FIG. 6A or 6B under the control of the main processor 140. Inaddition, the LED display unit 17 may turn on the LED at all times. Inaddition, in the pattern shown in FIG. 6A or 6B, time T1 at which theLED is turned on and time T2 at which the LED is turned off arearbitrary. For example, the times T1 and T2 may be set to equal to orless than 100 milliseconds. In this case, even when the LED isrepeatedly turned on and turned off, the LED can be visually perceivedby a user so as to be continuously turned on. In addition, the LEDdisplay unit 17 may control a ratio of the time T1 to the time T2 tothereby perform PWM control on the luminance of the LED.

Referring back to FIG. 4, the six-axis sensor 111 is a motion sensor(inertial sensor) including a three-axis acceleration sensor and athree-axis gyro (angular velocity) sensor. An inertial measurement unit(IMU) in which the above-mentioned sensors are modularized may beadopted as the six-axis sensor 111.

The magnetic sensor 113 is, for example, a three-axis geomagneticsensor.

The global positioning system (GPS) 115, including a GPS antenna notshown in the drawing, receives a wireless signal transmitted from a GPSsatellite to thereby detect coordinates of the current position of thecontrol device 10.

The six-axis sensor 111, the magnetic sensor 113, and the GPS 115 outputa detected value to the main processor 140 in accordance with a samplingcycle which is designated in advance. Alternatively, the six-axis sensor111, the magnetic sensor 113, and the GPS 115 output a detected value tothe main processor 140 at a timing which is designated by the mainprocessor 140, in accordance with a request of the main processor 140.

The communication unit 117 performs wireless communication with anexternal device. The communication unit 117 is configured to include anantenna, an RF circuit, a base band circuit, a communication controlcircuit, and the like, or is constituted by a device in which theantenna and the circuits are integrated. The communication unit 117performs wireless communication based on a standard such as Bluetooth(registered trademark) or a wireless LAN (including Wi-Fi (registeredtrademark)).

The sound interface 182 is an interface that inputs and outputs a soundsignal. In the present embodiment, the sound interface 182 includes theconnector 46 (FIG. 1) which is provided in the connection cable 40. Thesound codec 180 is connected to the sound interface 182, and performsencoding and decoding of a sound signal which is input and outputthrough the sound interface 182. In addition, the sound codec 180 mayinclude an A/D converter that converts an analog sound signal into adigital sound data, and a D/A converter that performs the reverseconversion. For example, the HMD 100 of the present embodiment outputs asound through the right ear phone 32 and the left ear phone 34, andcollects a sound by the microphone 63. The sound codec 180 convertsdigital sound data which is output by the main processor 140 into ananalog sound signal, and outputs the converted data through the soundinterface 182. In addition, the sound codec 180 converts an analog soundsignal which is input to the sound interface 182 into digital sounddata, and outputs the converted signal to the main processor 140.

The external connector 184 is a connector for connection of an externaldevice communicating with the main processor 140. The external connector184 is an interface that connects, for example, an external device tothe main processor 140 and performs connection of the external device ina case where the debugging of a program to be executed by the mainprocessor 140 and the collection of logs of operation of the HMD 100 areperformed.

The external memory interface 186 is an interface capable of performingconnection of a portable memory device, and includes, for example, amemory card slot, which is mounted with a card type recording medium soas to be capable of reading data, and an interface circuit. In thiscase, the size, shape, and standard of the card type recording mediumare not limited, and can be appropriately changed.

The universal serial bus (USB) connector 188 includes a connector and aninterface circuit based on a USB standard, and can be used to connect aUSB memory device, a smart phone, a computer, and the like. The versionof a USB standard which is appropriate for the size and shape of the USBconnector 188 can be appropriately selected and changed.

In addition, the HMD 100 includes a vibrator 19. The vibrator 19includes a motor (not shown), an eccentric rotor (not shown), and thelike, and generates vibration under the control of the main processor140. The HMD 100 generates vibration by the vibrator 19 with apredetermined vibration pattern, for example, in a case where anoperation with respect to the operation unit 110 is detected, in a casewhere the power supply of the HMD 100 is turned on or turned off, andthe like.

The sensor hub 192 and the FPGA 194 are connected to the image displayunit 20 through the interface (I/F) 196. The sensor hub 192 acquiresdetected values of the various sensors included in the image displayunit 20 and outputs the acquired values to the main processor 140. Inaddition, the FPGA 194 performs the processing of data transmitted andreceived between the main processor 140 and each unit of the imagedisplay unit 20 and the transmission of the data through the interface196.

Each of the right display unit 22 and the left display unit 24 of theimage display unit 20 is connected to the control device 10. As shown inFIG. 1, in the HMD 100, the connection cable 40 is connected to the leftholding portion 23, wirings connected to the connection cable 40 arelaid inside the image display unit 20, and each of the right displayunit 22 and the left display unit 24 is connected to the control device10.

The right display unit 22 includes a display unit substrate 210. Thedisplay unit substrate 210 is mounted with an interface (I/F) 211 whichis connected to the interface 196, a reception unit (Rx) 213 thatreceives data which is input from the control device 10 through theinterface 211, and an EEPROM 215 (storage unit).

The interface 211 connects the reception unit 213, the EEPROM 215, thetemperature sensor 217, the camera 61, the illuminance sensor 65, andthe LED indicator 67 to the control device 10.

The electrically erasable programmable read-only memory (EEPROM) 215stores various pieces of data so as to be capable of being read by themain processor 140. The EEPROM 215 stores, for example, data regardinglight emission characteristics and display characteristics of the OLEDunits 221 and 241 included in the image display unit 20, data regardingcharacteristics of the sensor included in the right display unit 22 orthe left display unit 24, and the like. Specifically, the EEPROM storesparameters related to gamma correction of the OLED units 221 and 241,data for compensating for detected values of the temperature sensors 217and 239, and the like. These pieces of data are generated by examinationduring the shipment of the HMD 100 from a plant and are written in theEEPROM 215. After the shipment, the main processor 140 can performprocessing using the data of the EEPROM 215.

The camera 61 performs imaging in response to a signal which is inputthrough the interface 211, and outputs captured image data or a signalindicating an imaging result to the control device 10.

The illuminance sensor 65 is provided at the end ER of the front frame27 as shown in FIG. 1, and is disposed so as to receive external lightfrom the front of a user wearing the image display unit 20. Theilluminance sensor 65 outputs a detected value corresponding to theamount of light received (light receiving intensity).

The LED indicator 67 is disposed in the vicinity of the camera 61 at theend ER of the front frame 27, as shown in FIG. 1. The LED indicator 67is turned on during imaging using the camera 61 to notify that imagingis being performed.

The temperature sensor 217 detects temperature and outputs a voltagevalue or a resistance value corresponding to the detected temperature,as a detected value. The temperature sensor 217 is mounted on the rearsurface side of the OLED panel 223 (FIG. 2). For example, thetemperature sensor 217 may be mounted on the same substrate as that ofthe OLED driving circuit 225. With such a configuration, the temperaturesensor 217 mainly detects the temperature of the OLED panel 223.

The reception unit 213 receives data which is transmitted by the mainprocessor 140 through the interface 211. In a case where the receptionunit 213 receives image data of an image displayed on the OLED unit 221,the reception unit outputs the received image data to the OLED drivingcircuit 225 (FIG. 2).

The left display unit 24 includes the display unit substrate 210. Thedisplay unit substrate 210 is mounted with an interface (I/F) 231 whichis connected to the interface 196, and a reception unit (Rx) 233 thatreceives data which is input from the control device 10 through theinterface 231. In addition, a six-axis sensor 235 and a magnetic sensor237 are mounted on the display unit substrate 210.

The interface 231 connects the reception unit 233, the six-axis sensor235, the magnetic sensor 237, and the temperature sensor 239 to thecontrol device 10.

The six-axis sensor 235 (display unit movement detection unit) is amotion sensor (inertial sensor) that includes a three-axis accelerationsensor and a three-axis gyro (angular velocity) sensor. An inertialmeasurement unit (IMU) in which the above-mentioned sensors aremodularized may be adopted as the six-axis sensor 235.

The magnetic sensor 237 is, for example, a three-axis geomagneticsensor.

The temperature sensor 239 detects temperature and outputs a voltagevalue or a resistance value corresponding to the detected temperature,as a detected value. The temperature sensor 239 is mounted on the rearsurface side of the OLED panel 243 (FIG. 2). For example, thetemperature sensor 239 may be mounted on the same substrate as that ofthe OLED driving circuit 245. With such a configuration, the temperaturesensor 239 mainly detects the temperature of the OLED panel 243.

In addition, the temperature sensor 239 may be built into the OLED panel243 or the OLED driving circuit 245. In addition, the above-mentionedsubstrate may be a semiconductor substrate. Specifically, in a casewhere the OLED panel 243 is mounted as an integrated circuit on anintegrated semiconductor chip together with the OLED driving circuit 245and the like as an Si-OLED, the temperature sensor 239 may be mounted onthe semiconductor chip.

The camera 61, the illuminance sensor 65, and the temperature sensor 217which are included in the right display unit 22 and the six-axis sensor235, the magnetic sensor 237, and the temperature sensor 239 which areincluded in the left display unit 24 are connected to the sensor hub192. The sensor hub 192 performs setting of a sampling cycle of eachsensor and initialization under the control of the main processor 140.

The sensor hub 192 performs electrical conduction to each sensor, thetransmission of control data, the acquisition of a detected value, andthe like in accordance with a sampling cycle of each sensor. Inaddition, the sensor hub 192 outputs detected values of the sensorsincluded in the right display unit 22 and the left display unit 24 tothe main processor 140 at a timing which is set in advance. The sensorhub 192 may have a function of temporarily holding the detected valuesof the sensors in accordance with a timing of output to the mainprocessor 140. In addition, the sensor hub 192 may have a function ofperforming conversion into data in a united data format in response todifferences between signal formats of output values of the respectivesensors or data formats thereof and outputting the converted data to themain processor 140.

In addition, the sensor hub 192 starts and stops electrical conductionto the LED indicator 67 under the control of the main processor 140, andturns on or turns off the LED indicator 67 in accordance with timings atwhich the camera 61 starts and terminates imaging.

The control device 10 includes a power supply unit 130, and operates bypower supplied from the power supply unit 130. The power supply unit 130includes a battery 132 which is rechargeable, and a power supply controlcircuit 134 that detects a remaining amount of the battery 132 andcontrols the charging of the battery 132. The power supply controlcircuit 134 is connected to the main processor 140, and outputs adetected value of the remaining amount of the battery 132 or a detectedvalue of a voltage to the main processor 140. In addition, power may besupplied from the control device 10 to the image display unit 20 on thebasis of power supplied by the power supply unit 130. In addition, aconfiguration may also be adopted in which the supply of power from thepower supply unit 130 to each unit of the control device 10 and theimage display unit 20 can be controlled by the main processor 140.

FIG. 5 is a functional block diagram showing the storage unit 122 andthe control unit 150 which constitute a control system of the controldevice 10. The storage unit 122 shown in FIG. 5 is a logical storageunit constituted by the non-volatile storage unit 121 (FIG. 4), and mayinclude the EEPROM 215. In addition, the control unit 150 and variousfunctional units included in the control unit 150 are formed bycooperation of software and hardware by the main processor 140 executingprograms. The control unit 150 and the functional units constituting thecontrol unit 150 are constituted by, for example, the main processor140, the memory 118, and the non-volatile storage unit 121.

The control unit 150 performs various processes using data stored in thestorage unit 122 and controls the HMD 100.

The storage unit 122 stores various pieces of data processed by thecontrol unit 150. The storage unit 122 stores setting data 123, contentdata 124, operation setting data 125, and display control data 126. Thesetting data 123 includes various setting values related to theoperation of the HMD 100. In addition, parameters, a matrix expression,an arithmetic expression, a lookup table (LUT), and the like may beincluded in the setting data 123 when being used by the control unit 150at the time of controlling the HMD 100.

The content data 124 is data of contents including an image or a videowhich is displayed by the image display unit 20 under the control of thecontrol unit 150, and includes image data or video data. In addition,the content data 124 may include sound data. In addition, the contentdata 124 may include image data of a plurality of images. In this case,the plurality of images are not limited to images that aresimultaneously displayed by the image display unit 20.

In addition, the content data 124 may be bidirectional type contents inwhich a user's operation is received by the control device 10 at thetime of displaying contents on the image display unit 20 and the controlunit 150 performs processing in accordance with the received operation.In this case, the content data 124 may include image data of a menuscreen displayed in a case where an operation is received, data fordetermining processing corresponding to an item included in the menuscreen, and the like.

The operation setting data 125 is data which is used in a process inwhich the control unit 150 detects a motion operation of moving thecontrol device 10. The operation setting data 125 includes data fordetermining individual motions with respect to patterns of one or aplurality of movements that are set in advance, as motions of the case10A. Specifically, the operation setting data includes a detected valueof a movement detected by the six-axis sensor 111 or data regarding achange in the detected value for each type of motion. This data mayinclude a plurality of detected values of the six-axis sensor 111, ormay be data regarding a feature amount obtained by an arithmeticoperation from the detected values. The operation setting data 125 maybe data which is used to be capable of determining whether or not thecase 10A has performed movement corresponding to a motion which is set,from the detected values of the six-axis sensor 111. More preferably,the operation setting data is data which is used to be capable ofspecifying a motion corresponding to the movement of the case 10A whichis detected by the six-axis sensor 111, among the plurality of motionswhich are set.

The display control data 126 is data which is used in a process ofchanging a display state of an image displayed by the image display unit20, in response to the motion of the case 10A. In more detail, thedisplay control data includes data indicating a mode for changing adisplay state in association with the set motion of the case 10A. In acase where the control unit 150 detects and specifies the motion of thecase 10A on the basis of the operation setting data 125, the controlunit changes the display of the image display unit 20 in a mode which isset in the display control data 126 in association with the specifiedmotion.

The control unit 150 has functions of the operating system (OS) 143, animage processing unit 145, a display control unit 147, an imagingcontrol unit 149, and an operation detection control unit 152. Thefunction of the operating system 143 is a function of a control programstored in the storage unit 122, and the functions of the other units arefunctions of application programs executed on the operating system 143.

The image processing unit 145 generates a signal transmitted to theright display unit 22 and the left display unit 24 on the basis of imagedata of an image or a video which is displayed by the image display unit20. The signal generated by the image processing unit 145 may be avertical synchronization signal, a horizontal synchronization signal, aclock signal, an analog image signal, or the like.

In addition, the image processing unit 145 may perform a resolutionconversion process of converting the resolution of image data intoresolution suitable for the right display unit 22 and the left displayunit 24 as necessary. In addition, the image processing unit 145 mayperform an image adjustment process of adjusting the luminance or chromaof image data, a 2D/3D conversion process of creating 2D image data from3D image data or creating 3D image data from 2D image data, and thelike. In a case where the image processing unit 145 performs suchimaging processing, the image processing unit generates a signal fordisplaying an image on the basis of processed image data and transmitsthe generated signal to the image display unit 20 through the connectioncable 40.

The image processing unit 145 may be constituted by hardware (forexample, a digital signal processor (DSP)) which is different from themain processor 140, in addition to being realized by the main processor140 executing a program.

The display control unit 147 generates a control signal for controllingthe right display unit 22 and the left display unit 24, and controls thegeneration and emission of image light which are performed by each ofthe right display unit 22 and the left display unit 24 in response tothe control signal. Specifically, the display control unit 147 controlsthe OLED driving circuits 225 and 245 to thereby perform the display ofan image by the OLED panels 223 and 243. The display control unit 147performs the control of a timing drawn on the OLED panels 223 and 243 bythe OLED driving circuits 225 and 245, the control of the luminance ofthe OLED panels 223 and 243, and the like on the basis of a signal whichis output by the image processing unit 145.

In addition, in a case where the operation detection control unit 152detects the motion of the control device 10, the display control unit147 changes a display mode of an image displayed on the image displayunit 20, or the like in accordance with the display control data 126.

The imaging control unit 149 controls the camera 61 to generate capturedimage data by imaging, and temporarily stores the generated image datain the storage unit 122. In addition, in a case where the camera 61 isconfigured as a camera unit including a circuit generating capturedimage data, the imaging control unit 149 acquires the captured imagedata from the camera 61 and temporarily stores the acquired image datain the storage unit 122.

The operation detection control unit 152 detects operations in the trackpad 14 and the operation unit 110, and outputs data corresponding to theoperations. For example, in a case where the buttons of the operationunit 110, or the like are operated, the operation detection control unit152 generates operation data indicating operation contents and outputsthe generated operation data to the display control unit 147. Thedisplay control unit 147 changes a display state in accordance with theoperation data which is input from the operation detection control unit152.

The operation detection control unit 152 detects the operation of asoftware button while the LED display unit 17 is being turned on. Theoperation detection control unit 152 detects a contact operation withrespect to the position of a mark displayed by the turn-on of the LEDdisplay unit 17 as the operation of a button (software operator) towhich a function is allocated in advance, while the LED display unit 17is being turned on.

For example, in the example of FIG. 1, an operation of coming intocontact with the position of the mark ∘ (circle) is detected as theoperation of the home button which is allocated to the position of themark ∘ (circle) in advance. In this case, the operation detectioncontrol unit 152 makes a display screen of the image display unit 20 andan operation state of the control device 10 transition to a basic stateof the operating system 143.

In addition, for example, in the example of FIG. 1, an operation ofcoming into contact with the position of the mark □ (quadrangle) isdetected as the operation of the history button which is allocated tothe position of the mark □ (quadrangle). In this case, the operationdetection control unit 152 displays a display screen of the imagedisplay unit 20 or a screen indicating the past operation state of thecontrol device 10 by the image display unit 20.

The operation detection control unit 152 specifies the movement of thecase 10A on the basis of a detected value of the six-axis sensor 111,and determines whether or not the movement of the case 10A correspondsto a motion which is set in advance. In addition, in the presentembodiment, the operation detection control unit 152 performs a processof specifying a motion corresponding to the movement of the case 10A. Inthis process, the operation detection control unit 152 acquires adetected value of the six-axis sensor 111. In addition, the operationdetection control unit may acquire a plurality of detected valuesdetected by the six-axis sensor 111. For example, in a case where thesix-axis sensor 111 performs detection with a predetermined cycle, theoperation detection control unit may continuously acquire the pluralityof detected values that are successively detected by the six-axis sensor111. The operation detection control unit 152 may store the detectedvalues of the six-axis sensor 111 in the memory 118 or the non-volatilestorage unit 121 as necessary. For example, a ring buffer may be formedin the memory 118 or the non-volatile storage unit 121, and theoperation detection control unit 152 may acquire a detected value of thesix-axis sensor 111 at all times and may store the acquired detectedvalue in the ring buffer. In this case, the number of latest detectedvalues corresponding to the capacity of the ring buffer is stored in thering buffer. The operation detection control unit 152 can perform theabove-described process on the basis of the detected values stored inthe ring buffer and the operation setting data 125.

In addition, in a case where the operation detection control unit 152detects an operation in the track pad 14, the operation detectioncontrol unit acquires coordinates of an operation position in anoperation detection region (detection region) of the track pad 14.

The HMD 100 may include an interface (not shown) for connection ofvarious external devices which are sources supplying contents. Forexample, the interface may be an interface, such as a USB interface, amicro USB interface, an interface for a memory card, which correspondsto wired connection, or may be constituted by a wireless communicationinterface. In this case, the external device is an image supply devicethat supplies an image to the HMD 100. A personal computer (PC), amobile phone terminal, a portable game machine, or the like is used asthe external device. In this case, the HMD 100 can output an image or asound based on content data which is input from the external devices.

FIGS. 7A and 7B are diagrams showing an operation example using thecontrol device 10, and specifically show an example of a motion ofmoving the case 10A. Both FIGS. 7A and 7B show a state where a userholds the case 10A in his or her hand. FIG. 7A is a front view when thecontrol device 10 is seen from the user side, and FIG. 7B is a sideview. Meanwhile, the connection cable 40 is not shown in FIGS. 7A and7B.

In the present embodiment, six motions are specified in advance as modesof an operation of moving the control device 10.

FIG. 7A shows four motions A, B, C, and D. FIG. 7B shows two motions Eand F.

In the example shown in the drawing, a user grasps the case 10A with hisor her right hand RH, but may grasp the case with his or her left hand.In addition, the user may hold the case 10A in a join manner using a jigor the like.

The motion A is an operation of rotating the control device 10 clockwise(CW) on the basis of the right and left of the user's body, as indicatedby an arrow A in the drawing. The six-axis sensor 111 detects anacceleration and an angular velocity which indicate the clockwisemovement of the case 10A. The motion B is an operation of rotating thecontrol device 10 counterclockwise (CCW) as indicated by an arrow B inthe drawing. The motion A can be referred to as a right rotationoperation, and the motion B can be referred to as a left rotationoperation.

A right-left direction in a case where a user uses the case by holdingthe case in his or her hand, as shown in the drawing, is set in thecontrol device 10 in advance. For example, as shown in FIGS. 7A and 7B,in a case where the user holds the case 10A in his or her hand, thedirection may be specified in advance such that an upper end of the case10A faces upward and a surface having the track pad 14 disposed thereonis directed to the user's right side. Meanwhile, the upper end face ofthe case 10A refers to a face on which the LED indicator 12 and thepower supply switch 18 are provided. In addition, a configuration may beadopted in which the user can set surfaces of the case 10A which are setto be the right side and the left side, in a case where the controldevice 10 is held in the user's hand, by operating the HMD 100. In thiscase, the operation setting data 125 includes data in which a detectiondirection of the six-axis sensor 111 and the right and left of theuser's body are associated with each other.

The motion C is an operation of moving the control device 10 in such amanner as to draw an arc rightward as indicated by an arrow C in thedrawing, and the motion D is an operation of moving the control device10 in such a manner as to draw an arc leftward as indicated by an arrowD in the drawing. The motion C can be referred to as a rightwardmovement operation, and the motion D can be referred to as a leftwardmovement operation.

Meanwhile, as indicated by an arrow C′ in the drawing, a configurationmay also be adopted in which an operation of linearly moving the controldevice 10 to the right is detected as the motion C. Similarly, asindicated by an arrow D′ in the drawing, a configuration may also beadopted in which an operation of linearly moving the control device 10to the left is detected as the motion D.

The motion E is an operation of moving the control device 10 in such amanner as to draw an arc upward as indicated by an arrow E in FIG. 7B.The motion F is an operation of moving the control device 10 in such amanner as to draw an arc downward as indicated by an arrow F in FIG. 7B.The motion E can be referred to as a swing-up operation, and the motionD can be referred to as a swing-down operation. As indicated by an arrowE′ in the drawing, a configuration may also be adopted in which anoperation of linearly moving the control device 10 upward is detected asthe motion E. In addition, as indicated by an arrow F′ in the drawing, aconfiguration may also be adopted in which an operation of linearlymoving the control device 10 downward is detected as the motion E.

Some of the six motions are associated with each other as paired motions(movements). The motion A and the motion B are set as paired motions.Similarly, the motion C, the motion D, the motion E, and the motion Fare set as paired motions. Hereinafter, paired motions are referred toas reverse motions. For example, the motion B is equivalent to thereverse motion of the motion A, and vice versa. The same is true of themotions C, D, E, and F.

Any one of the paired motions is equivalent to an operation (movement,motion) in a first movement direction, and the other is equivalent to anoperation in a second movement direction. The first movement directionand the second movement direction are not limited to oppositedirections. The directions may be directions in which operations can bedetected as movements in different directions by the six-axis sensor 111or the magnetic sensor 113 and/or the camera 61 which are used insteadof the six-axis sensor 111. Each of the motions A to F, C′, D′, E′, andF′ is set as a movement in a specific direction, but a movement close tothe specific direction may be regarded as a movement in the specificdirection. For example, the motion C shown in FIG. 7A is a movement inthe horizontal direction in the drawing, but a movement in a range of anangle of approximately five degrees may be detected as the motion C. Arange in which motions are regarded as motions in the same direction maybe arbitrarily set, and may be set to, for example, ±5 degrees or ±10degrees. The same is true of the first movement direction and the secondmovement direction.

The operation detection control unit 152 detects six motions A to F ofthe case 10A, and the display control unit 147 controls the display ofthe image display unit 20 so as to correspond to the motion detected bythe operation detection control unit 152.

FIGS. 8A to 8D, FIGS. 9A to 9C, and FIGS. 10A to 10C are diagramsshowing changes in display corresponding to the operation of the HMD.

FIGS. 8A to 8D show examples in which display is enlarged and reduced.FIG. 8A shows a display example before enlargement, FIG. 8B shows adisplay example after enlargement, FIG. 8C shows a display examplebefore reduction, and FIG. 8D shows a display example after reduction. Asign VR in the drawing indicates a user's field of view, and a sign Vindicates a display region in which an image can be displayed by theright display unit 22 and the left display unit 24. Signs W1, W2, and W3indicate a window as an example of an image displayed in the displayregion V.

In a case where display is enlarged as indicated by an arrow in thedrawing in response to the motion of the case 10A from the state shownin FIG. 8A, the display control unit 147 determines a center position CEof an enlargement process in a display region V. The center position CEmay be designated in accordance with an operation in the track pad 14,or may be automatically determined on the basis of the position of apointer (not shown) and the like at the time of starting the enlargementprocess. In the example of FIG. 8B, a window W1 of the display region Vshown in FIG. 8A is displayed in an enlarged manner centering on thecenter position CE. Comparing the display example of FIG. 8A with thedisplay example of FIG. 8B, an enlargement ratio of the window W1 may bedetermined on the basis of an operation with respect to the controldevice 10 as described later with reference to a flow chart, or may beset in advance.

In addition, also in a case where display is reduced as indicated by anarrow in the drawing in response to the motion of the case 10A from thestate shown in FIG. 8C, the display control unit 147 determines a centerposition CE of a reduction process in a display region V. The centerposition CE may be designated in accordance with an operation in thetrack pad 14, or may be automatically determined on the basis of theposition of a pointer (not shown) and the like at the time of startingthe enlargement process. In the example of FIG. 8D, a window W1 of thedisplay region V shown in FIG. 8C is displayed in a reduced mannercentering on the center position CE. Comparing the display example ofFIG. 8C with the display example of FIG. 8D, a reduction ratio of thewindow W1 may be determined in accordance with an operation with respectto the control device 10 as described later with reference to a flowchart, or may be set in advance.

FIGS. 9A to 9C show an example in which display is scrolled. FIG. 9Ashows a display example before scrolling, FIG. 9B shows a displayexample in which scrolling is performed upward, and FIG. 9C shows adisplay example in which scrolling is performed downward.

The display control unit 147 can scroll the display up and down inresponse to the motion of the case 10A, as indicated by an arrow in thedrawing, from the state shown in FIG. 9A. In a case where scrolling isperformed upward, a display position of a window W1 is moved upward asshown in FIG. 9B. In addition, in a case where scrolling is performeddownward, a display position of the window W1 is moved downward as shownin FIG. 9C. As shown in the display examples of FIGS. 9A to 9C, theamount of scrolling of the window W1 may be determined in accordancewith an operation with respect to the control device 10 as describedlater with reference to a flow chart, or may be set in advance.

Although not shown in the drawing, the display control unit 147 mayperform horizontal scrolling of moving an image displayed in the displayregion V in the left direction or the right direction.

Any one of upward scrolling of FIG. 9B and downward scrolling of FIG. 9Cis equivalent to a first display operation direction, and the other isequivalent to a second display operation direction. These scrollingoperations are just examples, and the first display operation directionand the second display operation direction may be different directionsof the movement of display. For example, a sideways (may be eitherrightward or leftward) scrolling direction and a vertical (may be eitherupward or downward) scrolling direction may be set to be the firstdisplay operation direction and the second display operation direction.In addition, the movement of display in an oblique direction may be setto be movement in the first display operation direction and/or thesecond display operation direction. That is, movements of display in twodirections capable of recognized to be different directions by a usercan be set to be movement in the first display operation direction andthe second display operation direction.

In addition, FIGS. 9B and 9C show an example in which the upwardscrolling of display is performed in response to an operation of movingthe case 10A upward, and the downward scrolling of display is performedin response to an operation of moving the case 10A downward. Thesescrolling operations are just examples, and the movement direction ofthe case 10A which is detected by the six-axis sensor 111 and themovement direction of display do not have to be relatively set as thesame direction with respect to a user. The direction of a movementdetected by the six-axis sensor 111 and the movement direction ofdisplay may be associated with each other in advance, or may be, forexample, opposite directions.

FIGS. 10A to 10C show an example in which display is changed over. FIG.10A shows a display example before processing, FIG. 10B shows a displayexample in which display is shifted to the previous screen, and FIG. 10Cshows a display example in which display is shifted to the next screen.

The examples shown in FIGS. 10A to 10C correspond to an example in whichthe content data 124 displayed by the display control unit 147 includesa plurality of images of which the display order is set, and the displaycontrol unit 147 changes over and displays images in accordance with thedisplay order during the reproduction of the content data 124. Theseimages are not limited to images such as a single photograph, and maybe, for example, a combination of images or texts constituting onescreen. Specifically, in a configuration in which the control unit 150executes an application program on the operating system 143, a web pagedisplayed by a web browser is an example of an image to be displayed. Inaddition, in a configuration in which a presentation program is executedas an application program, a slide sheet for presentation which includesa plurality of slide sheets is an example of the content data 124including a plurality of images.

FIG. 10A shows an example in which a slide sheet for presentation isdisplayed, as an example. For example, in FIG. 10A, a window W1including the slide sheet is displayed. In this state, when the displaycontrol unit 147 shifts a screen to the previous screen, a window W2including the previous slide sheet is displayed in a display region V.In FIG. 10A, when the display control unit 147 shifts a screen to thenext screen, a window W3 including the next slide sheet is displayed ina display region V.

In other words, the examples of FIGS. 10A to 10C show the display changeof images being changed over in order and the display change of imagesbeing changed over in reverse order in an operation of displaying aplurality of images of which the display order is set.

In the present embodiment, setting is performed such that the displaycontrol unit 147 enlarges display in response to a motion A (clockwiserotation) and reduces display in response to a motion B(counterclockwise rotation). In addition, the display control unit 147scrolls display upward in response to a motion E (swing-up), and scrollsdisplay downward in response to a motion F (swing-down). In addition,the display control unit 147 shifts display to the next screen inresponse to a motion C (clockwise movement) and shifts display to thenext screen in response to a motion D (counterclockwise movement).

FIGS. 11 and 12 are flow charts showing the operation of the HMD 100.

When a motion operation of moving the control device 10 is started (stepS11), the control unit 150 determines whether to use a positionindication (position input) using the track pad 14 (step S12). In a casewhere a position indication is used, a center position CE is designatedin accordance with the operation of the track pad 14 in a process ofenlarging display during the execution of a motion A and a process ofreducing display during the execution of a motion B. Whether to use aposition indication may be designated by a user through an inputoperation with respect to the operation unit 110, or may be set inadvance.

In a case where a position indication is used (step S12; Yes), thecontrol unit 150 proceeds to step S31 (FIG. 12). This case will bedescribed later.

In a case where a position indication is not used (step S12; No), thecontrol unit 150 starts acquiring a detected value of the six-axissensor 111 (step S13). Thereafter, the control unit 150 acquires adetected value of the six-axis sensor 111 with a cycle which is set inadvance.

Subsequently, the control unit 150 determines whether or not anoperation serving as a trigger for controlling display is performed(step S14). In a case where an operation is not performed (step S14;No), the control unit stands by until an operation is performed. Anoperation with respect to various operators such as the buttons 11provided in the case 10A is an example of the operation serving as atrigger. In the present embodiment, a contact operation with respect tothe track pad 14 is set as a trigger.

When the control unit 150 determines that a contact operation withrespect to the track pad 14 has been performed (step S14; Yes), thecontrol unit determines whether or not an elapsed time since the startof acquisition of detected values in step S13 has reached a time whichis set (set time) (step S15).

A function is allocated in advance to an operator corresponding to anoperation serving as a trigger for controlling display. For example, inthe present embodiment, a contact operation with respect to the trackpad 14 is set as a trigger operation, but a position input operation anda function of a software button corresponding to the display position ofthe LED display unit 17 are allocated to the track pad 14. The operationdetection control unit 152 can change over the detection as a triggeroperation or the detection of a position input operation of the trackpad 14 or an operation of a software button at a display position of theLED display unit 17 in a case where a contact operation with respect tothe track pad 14 is performed. In a case where a time elapsed from thestart of acquisition of detected values of the six-axis sensor 111 tothe operation of the track pad 14 has reached a set time (step S15;Yes), the operation detection control unit 152 performs a normaloperation mode without receiving a motion operation. The operationdetection control unit 152 clears the detected values of the six-axissensor 111 which is acquired in advance and is stored (step S16), and afunction allocated as a normal operation is performed on the triggeroperation detected in step S14 (step S17), thereby terminating thisprocess.

For example, in a case where a contact at the position of the mark ∘(circle) of the track pad 14 is detected in step S14, the operationdetection control unit 152 performs control corresponding to theoperation of the home button in step S17.

In a case where the elapsed time has not reached the set time (step S15;No), the operation detection control unit 152 starts a display operationmode (step S18). The display operation mode is an operation mode inwhich a trigger operation is detected as a trigger of a motionoperation. The operation detection control unit 152 analyzes themovement of the case 10A on the basis of the detected values of thesix-axis sensor 111 (step S19). The detected values analyzed in step S19includes the past detected values which are started to be acquired instep S13 and are stored in the memory 118 or the non-volatile storageunit 121, and particularly, includes detected values acquired until anoperation serving as a trigger is detected in step S14.

For example, in a case where a user moves the control device 10 whileholding the control device in his or her hand and touches the track pad14 during the movement of the case 10A, the control unit 150 analyzesthe movement of the case 10A from before the contact operation withrespect to the track pad 14. For example, even when a timing of anoperation of moving the control device 10 and a timing of an operationserving as a trigger deviate from each other against the intention of auser, the movement of the case 10A which is performed by the user'sintention is analyzed.

The operation detection control unit 152 determines whether or notpreset motions (for example, motions A to F) include a motioncorresponding to the movement of the case 10A which is analyzed in stepS19 (step S20). In a case where there is no corresponding motion (stepS20; No), the operation detection control unit 152 analyzes the movementof the case 10A again (step S21), and returns to step S20 to performdetermination. In step S21, the analysis is performed inclusive ofdetected values of the six-axis sensor 111 which are acquired during aperiod between step S19 and step S21, and thus there is the possibilitythat a new motion of the case 10A can be detected.

In a case where there is a motion corresponding to the movement of thecase 10A (step S20; Yes), the operation detection control unit 152determines whether to perform a combination operation (step S22). Thecombination operation refers to an operational method of controllingdisplay by combining a plurality of paired motions. Whether to perform acombination operation may be determined in accordance with an operationperformed on the control device 10 or may be set in advance.

In a case where the combination operation is not performed (step S23;No), the display control unit 147 determines contents of display changecorresponding to the motion and the amount of display change which aredetermined by the operation detection control unit 152 in step S20 (stepS23). For example, an enlargement ratio and a reduction ratio aredetermined in response to the motions A and B, and a scrolling directionand the amount of scrolling are determined in response to the motions Eand F. The amount of change in display can be set to be a preset amountfor each type of motion.

The display control unit 147 changes the display of the image displayunit 20 in accordance with the contents of display control and theamount of display change which are determined in step S23 (step S24),and proceeds to step S29 to be described later.

In a case where the combination operation is performed (step S23; Yes),the display control unit 147 starts display change corresponding to themotion determined by the operation detection control unit 152 in stepS20 (step S25). After the display change is started, the operationdetection control unit 152 analyzes the movement of the case 10A on thebasis of detected values of the six-axis sensor 111 (step S26). Theoperation detection control unit 152 determines whether or not themovement of the case 10A which corresponds to the reverse motion of themotion determined in step S20 has been performed (step S27). While themovement of the reverse motion is not performed (step S27; No), theoperation detection control unit performs the analysis of steps S26 andS27 and stands by.

In a case where the operation detection control unit 152 determines thatthe movement of the reverse motion has been performed (step S27; Yes),the display control unit 147 stops the display change (step S28) andproceeds to step S29.

For example, in a case where the control unit 150 starts upwardscrolling in response to the motion E in step S25 and then determinesthat the motion F has been performed in step S27, the control unit stopsthe upward scrolling. In this manner, since a timing of the stop ofchange in display corresponding to a motion can be designated by areverse motion, a user can freely adjust the amount of change indisplay.

In step S29, the operation detection control unit 152 determines whetheror not the trigger operation has been canceled (step S29). In a casewhere the trigger operation has not been canceled (step S29; No), thisprocess returns to step S20. In a case where the operation of anoperator which is allocated to the trigger operation has been canceled(step S29; Yes), this process is terminated.

On the other hand, in a case where a position indication is used (stepS12; Yes), the operation detection control unit 152 determines whetheror not a position indication operation using the track pad 14 has beenperformed (step S31). In a case where the position indication operationhas not been performed (step S31; No), the operation detection controlunit stands by until the position indication operation is performed.When the operation detection control unit 152 determines that theposition indication operation using the track pad 14 has been performed(step S31; Yes), the operation detection control unit determines whetheror not an elapsed time of standby of the position indication operationin step S31 has reached a time which is set (set time) (step S32).

In a case where the operation detection control unit 152 determines thatthe standby time of the position indication operation has reached theset time (step S32; Yes), the operation detection control unit performsa normal operation mode without receiving a motion operation. Theoperation detection control unit 152 performs the operations of stepsS16 and S17 mentioned above, thereby terminating this process.

In a case where the elapsed time has not reached the set time (step S32;No), the operation detection control unit 152 starts a display operationmode (step S33). The operation detection control unit 152 analyzes themovement of the case 10A on the basis of detected values of the six-axissensor 111 (step S34). In step S33, the operation detection control unit152 may perform analysis inclusive of the past detected values whichhave been acquired in advance and are stored in the memory 118 or thenon-volatile storage unit 121.

The operation detection control unit 152 determines whether or notpreset motions (for example, motions A to F) include a motioncorresponding to the movement of the case 10A which is analyzed in stepS34 (step S35). In a case where there is no corresponding motion (stepS35; No), the operation detection control unit 152 analyzes the movementof the case 10A again (step S36), and returns to step S35 to performdetermination.

In a case where there is a motion corresponding to the movement of thecase 10A (step S35; Yes), the operation detection control unit 152acquires coordinates of an indication position of the positionindication operation which is detected in step S31 (step S37). Here, theoperation detection control unit 152 may obtain display positions of theright display unit 22 and the left display unit 24 which correspond tothe acquired indication position.

The operation detection control unit 152 determines whether to perform acombination operation (step S38). The combination operation is asdescribed above. In a case where the combination operation is notperformed (step S38; No), the display control unit 147 determinescontents of display change corresponding to the motion and the amount ofdisplay change which are determined by the operation detection controlunit 152 in step S36, similar to step S23 (step S39).

The display control unit 147 changes the display of the image displayunit 20 in accordance with the contents of display change and the amountof display change which are determined in step S39, on the basis of theindication position acquired in step S37 by the operation detectioncontrol unit 152 (step S40), and proceeds to step S45 to be describedlater.

In a case where the combination operation is performed (step S38; Yes),the display control unit 147 starts display change corresponding to themotion determined by the operation detection control unit 152 in stepS36 (step S41). In step S41, the indication position acquired by theoperation detection control unit 152 in step S37 is set to be thestandard of display change.

In steps S39 and S41, the display control unit 147 enlarges and reducesdisplay with the indication position as a center position CE, forexample, as shown in FIGS. 8A to 8D.

After the display change is started, the operation detection controlunit 152 analyzes the movement of the case 10A on the basis of detectedvalues of the six-axis sensor 111 (step S42). The operation detectioncontrol unit 152 determines whether or not the movement of the case 10Awhich corresponds to the reverse motion of the motion determined in stepS20 has been performed (step S43). While the movement of the reversemotion is not performed (step S43; No), the operation detection controlunit performs the analysis of steps S42 and S43 and stands by.

In a case where the operation detection control unit 152 determines thatthe movement of the reverse motion has been performed (step S43; Yes),the display control unit 147 stops the display change (step S44) andproceeds to step S45.

In step S45, the operation detection control unit 152 determines whetheror not the trigger operation has been canceled (step S45). In a casewhere the trigger operation has not been canceled (step S45; No), theprocess returns to step S35. In a case where the operation of anoperator which is allocated to the trigger operation has been canceled(step S45; Yes), this process is terminated.

Meanwhile, in a case where the trigger operation has been canceledbefore it is determined in steps S26 and S27 that the reverse motion hasbeen performed, the display control unit 147 may stop the display changein step S28 similar to a case where the reverse motion has beenperformed, thereby terminating this process.

Similarly, in a case where the trigger operation has been canceledbefore it is determined in steps S42 and S43 that the reverse motion hasbeen performed, the display control unit 147 may stop the display changein step S44 similar to a case where the reverse motion has beenperformed, thereby terminating this process.

Further, in addition to the reverse motion, a specific motion ofstopping display change may be set in advance. For example, the movementof the case 10A moving up and down little by little with a smalleramount of operation than the motions E and F may be set as a stop motionserving as a trigger of stopping display change. In this case, in stepsS27 and S43, the operation detection control unit 152 may determine thatthe reverse motion or the stop motion has been performed.

In addition, in step S31 described above, in a case where the operationdetection control unit 152 detects the first contact operation withrespect to the track pad 14, the operation detection control unit maydetermine that a position indication operation has been performed, butmay fixedly set an indication position in response to a user'sinstruction. For example, in a case where a contact operation withrespect to the track pad 14 has been continuously performed, it isdetermined that a position indication operation has been performed at atiming which is instructed by a user, and an indication position at thetiming may be acquired in step S37. A method of making a user give aninstruction of a timing may be, for example, an operation of an operatorother than the track pad 14 included in the control device 10. Inaddition, for example, a timing may be designated by the movement of theimage display unit 20. Specifically, the movement of the image displayunit 20 moving up and down little by little may be set as a motion of adisplay unit that designates a timing of a position indication. Themovement of the image display unit 20 can be detected and determined bythe operation detection control unit 152 on the basis of the detectedvalues of the six-axis sensor 235. In this case, a user can designate atiming by moving his or her head while operating the track pad 14.Accordingly, the user wearing the image display unit 20 on his or herhead, particularly, in front of his or her eyes can easily perform anoperation of designating any position without viewing the control device10. In addition, the user may display an image indicating an indicationposition based on the track pad 14, that is, a contact position, or thelike on the image display unit 20 during a period between the start ofthe user's contact operation with respect to the track pad 14 and thedesignation of a timing by the movement of the image display unit 20. Inthis case, the user can adjust an operation position of the track pad 14while viewing the display of the image display unit 20 and can fixedlyset any indication position by the movement of the head, and it ispossible to achieve a further improvement in operability.

As described above, the HMD 100 is a head-mounted display deviceincluding the image display unit 20 that makes a user visually perceivean image. The HMD 100 includes the control device 10 as an operationdevice capable of being moved independently of the image display unit20. The control device 10 includes the six-axis sensor 111 that detectsthe movement of the control device 10 and the track pad 14 that receivesan operation different from the detection of the six-axis sensor 111.The HMD 100 includes the control unit 150 that controls the display ofthe image display unit 20 in accordance with a movement detected by thesix-axis sensor 111 in a state where the track pad 14 receives anoperation. Thereby, a user can easily control the display of the imagedisplay unit 20 in accordance with an operation of moving the controldevice 10. In addition, it is possible to prevent a user's unintendedchange in display by the operation state of the track pad 14.

In addition, the operation detection control unit 152 detects theoperation of the track pad 14 on which a contact operation can beperformed, as a trigger of a motion operation. In addition, theoperation serving as a trigger may be set as an operation of a buttoncapable of being pressed. In any case, it is possible to increaseoperability in a case where the track pad 14 is operated while movingthe case 10A of the control device 10. A contact operation and apressing operation are operations capable of being performed by a userwearing the image display unit 20 without viewing the track pad 14 andthe button. For this reason, a user wearing the head-mounted HMD 100 caneasily and reliably perform an operation of changing display even whenthe user hardly visually perceives the control device 10.

In addition, the control unit 150 changes a display magnification of animage displayed on the image display unit 20 in a case where movementscorresponding to an operation of rotating a device by the six-axissensor 111, for example, the motions A and B are detected. Thereby, itis possible to change the display magnification in accordance with aninstinctive operation.

In addition, the control unit 150 changes an image displayed on theimage display unit 20 in a direction which is set in advance in a casewhere movements corresponding to an operation of lifting down a deviceby the six-axis sensor 111 or an operation of lifting up the device, forexample, the motions E, E′, F, and F′ are detected. Thereby, it ispossible to change the image displayed on the image display unit 20 inaccordance with an instinctive operation.

In addition, the control unit 150 may change the display of the imagedisplay unit 20 in a state where the track pad 14 receives an operationand may then fixedly set a change in the display of the image displayunit 20 in a case where the operation of the track pad 14 is canceled.Thereby, it is possible to perform an operation of fixedly setting achange in the display of the image display unit 20. For this reason, itis possible to increase the degree of freedom of a change in display andto achieve an improvement in operability.

In addition, a predetermined function which is performed in a case wherethe track pad 14 receives an operation is set in the control unit 150.For example, a function of a position input and a function of a softwarebutton corresponding to a display position of a mark during the turn-onof the LED display unit 17 are set. The operation detection control unit152 performs a normal mode and a display operation mode in a changeovermanner. The normal mode is an operation mode in which a predeterminedfunction is performed in a case where the track pad 14 receives anoperation. The display operation mode is an operation mode in which thedisplay of the image display unit 20 is controlled in accordance with amovement detected by the six-axis sensor 111 in a state where the trackpad 14 receives an operation. Thereby, an operator which is set as atrigger can be used as an operation unit that indicates a predeterminedfunction. For this reason, it is not necessary to provide a dedicatedoperator used as a trigger of a motion operation in the control device10, and thus there is an advantage in terms of a reduction in the sizeof the control device 10.

In addition, the control unit 150 performs the display operation mode ina case where the track pad 14 receives an operation within a set timeafter the six-axis sensor 111 detects the start of movement. Thereby, itis possible to easily select whether to perform the normal mode or toperform the display operation mode in accordance with the operation ofthe track pad 14.

In a case where the control unit 150 starts the display operation mode,the control unit reflects a detection history of the six-axis sensor 111until an operation of the track pad 14 is received after the six-axissensor 111 detects the start of movement, to thereby control the displayof the image display unit 20. Thereby, the movement of the controldevice 10 until the start of the display operation mode can be reflectedon the control of display.

In addition, the control device 10 includes the track pad 14 thatreceives a position indication operation, and receives an operation ofthe track pad 14 as a position indication operation.

In addition, the control unit 150 changes the display of the imagedisplay unit 20 on the basis of a display position corresponding to aposition indicated by the position indication operation received by thetrack pad 14 in a state where the track pad 14 receives an operation.Thereby, it is possible to control the display of the image display unit20 with a high level of freedom by using the position indicationoperation and the movement of the control device 10. In addition, it ispossible to perform complicated display control by a simple operationand to achieve an improvement in operability.

In addition, the control unit 150 displays an image displayed on theimage display unit 20 in an enlarged or reduced manner, centering on adisplay position corresponding to a position indicated by the positionindication operation received by the track pad 14. Thereby, it ispossible to realize control of enlarging or reducing display centeringon a user's desired position by a simple operation using the controldevice 10.

In addition, in the present embodiment, a trigger of a motion operationfunctions as the track pad 14 on which a position indication operationcan be performed. Thereby, operability in a case where the controldevice 10 is operated without being visually perceived is excellentbecause of a small number of objects to be operated, and thus it ispossible to easily and reliably operate the control device 10 even whena user wearing the head-mounted HMD 100 hardly visually perceives thecontrol device 10. In addition, this is advantageous in a case where thecontrol device 10 is made small.

In addition, the HMD 100 includes the six-axis sensor 235 that detectsthe movement of the image display unit 20. The control unit 150specifies an indication position of an operation received by the trackpad 14 at a timing when the six-axis sensor 235 detects a predeterminedmovement. Thereby, an operation of fixedly setting an indicationposition during a position indication operation can be performed by themovement of the image display unit 20. For this reason, it is possibleto further improve operability in a case where a user wearing thehead-mounted HMD 100 operates the control device 10.

In addition, the control unit 150 performs a first control ofcontrolling the display of the image display unit 20 in accordance withthe type of operation corresponding to movement detected by the six-axissensor 111 in a state where the track pad 14 receives an operation. Thefirst control relates to, for example, control of changing display inaccordance with the motions A to F, and particularly, corresponds to thecontrols of changing display without being accompanied by a positionindication operation in steps S23, S24, S39, and S40. In addition, thecontrol unit performs a second control of changing the display of theimage display unit 20 on the basis of a display position correspondingto a position indicated by the position indication operation received bythe track pad 14. The second control is equivalent to, for example, anoperation of enlarging or reducing display with an indication positionof the track pad 14 as a center position CE. In addition, the controlunit performs a third control of changing the display of the imagedisplay unit 20 in response to a direction of a movement detected by thesix-axis sensor 111. The third control is equivalent to upwardscrolling, downward scrolling, leftward scrolling, rightward scrolling,the changeover of images based on the display order, and the like. Inmore detail, the third control is equivalent to the controls using areverse motion in steps S25 to S28 and S41 to S44. The control unit 150can perform the first control, the second control, and the third controlin a changeover manner.

Thereby, it is possible to change the display of the image display unit20 in accordance with three controls, in response to the movement of thecontrol device 10.

In addition, the control unit 150 performs control of moving an imagedisplayed on the image display unit 20 in a first display operationdirection and a second display operation direction making a pair withthe first display operation direction. For example, the upward scrollingand the downward scrolling shown in FIGS. 9A to 9C are equivalent tomovements in the first display operation direction and the seconddisplay operation direction. In addition, the rightward and leftwardscrollings are equivalent to movements in the first display operationdirection and the second display operation direction. In addition,regarding the changeover of a plurality of images of which the displayorder is set, which are shown in FIGS. 10A to 10C, it can be said thatthe display change of the images being changed over in order and thedisplay change of the images being changed over in reverse order areequivalent to movements in the first display operation direction and thesecond display operation direction.

The first display operation direction is associated with a firstmovement direction detected by the six-axis sensor 111, and the seconddisplay operation direction is associated with a second movementdirection detected by the six-axis sensor 111. The first movementdirection and the second movement direction correspond to paired motionsamong the motions of the case 10A. For example, the motion E isassociated with upward scrolling, and the motion F is associated withdownward scrolling. Thereby, it is possible to move an image displayedon the image display unit 20 by moving the control device 10 and toindicate a direction of movement of the image in accordance with amovement direction of the control device 10.

In addition, the control unit 150 starts the movement of an image in thefirst display operation direction in response to the movement in thefirst movement direction which is detected by the six-axis sensor 111,and then stops the movement of the image in the first display operationdirection in a case where the movement thereof in the second movementdirection is detected by the six-axis sensor 111. Thereby, it ispossible to move the image displayed on the image display unit 20 to adesired position by combining the movements of the control device 10 inthe two directions.

Here, a movement detection unit that detects a movement in the firstmovement direction and/or a movement in the second movement directionmay be the magnetic sensor 113 without being limited to the six-axissensor 111. In addition, it is possible to detect the movement in thefirst movement direction and/or the movement in the second movementdirection by using an image captured by the camera 61. In this case, thecamera 61 is equivalent to the movement detection unit. In addition, themovement detection unit may include the function of the control unit 150that analyzes an image captured by the camera 61 to detect a movement.

In addition, the control unit 150 starts the movement of an image in thefirst display operation direction or the second display operationdirection in response to a movement detected by the six-axis sensor 111,and then stops the movement of the image on the image display unit 20 ina case where an operation received by the track pad 14 is canceled.Thereby, it is possible to move the image displayed on the image displayunit 20 to a desired position by combining the movement of the controldevice 10 and the operation of the track pad 14.

In addition, as described above, the HMD 100 is a head-mounted displaydevice including the image display unit 20 that makes a user visuallyperceive an image. The HMD 100 includes the control device 10 as anoperation device capable of being moved independently of the imagedisplay unit 20. The control device 10 includes the six-axis sensor 111that detects the movement of the control device 10. In addition, the HMD100 includes the control unit 150 that controls the display of the imagedisplay unit 20 in accordance with a movement detected by the six-axissensor 111. The control unit 150 performs a first display changecorresponding to a movement in a first movement direction which isdetected by the six-axis sensor 111 and a second display changecorresponding to a movement in a second movement direction which isdetected by the six-axis sensor 111, as a process performed on an imagedisplayed on the image display unit 20. The control unit starts thefirst display change in a case where the movement in the first movementdirection is detected, and then stops the first display change in a casewhere the movement in the second movement direction is detected by thesix-axis sensor 111. Thereby, it is possible to easily control thedisplay of the image display unit 20 in accordance with an operation ofmoving the control device 10. In addition, it is possible to move theimage displayed on the image display unit 20 to a desired position bycombining the movements of the control device 10 in the two directions.

Here, the movement detection unit that detects the movement in the firstmovement direction and/or the movement in the second movement directionmay be the magnetic sensor 113 without being limited to the six-axissensor 111. In addition, it is possible to detect the movement in thefirst movement direction and/or the movement in the second movementdirection by using an image captured by the camera 61. In this case, thecamera 61 is equivalent to the movement detection unit. In addition, themovement detection unit may include the function of the control unit 150that analyzes an image captured by the camera 61 to detect a movement.

In addition, the control unit 150 performs the first display change in acase where the movement in the first movement direction is detected bythe six-axis sensor 111, and performs the second display change in acase where the movement in the second movement direction is detected bythe six-axis sensor 111. Thereby, it is possible to move an imagedisplayed on the image display unit 20 by moving the control device 10and to indicate a direction of movement of the image in accordance witha movement direction of the control device 10.

For example, the control unit 150 can perform upward scrolling as thefirst display change in a case where the motion E is detected as themovement in the first movement direction, and can perform downwardscrolling as the second display change in a case where the motion F isdetected as the movement in the second movement direction. In a casewhere the control unit starts the upward scrolling in response to thedetection of the motion E and then detects the motion F as the movementin the second movement direction, the control unit can also stop theupward scrolling.

In addition, the control unit 150 changes the display of the imagedisplay unit 20 in accordance with a movement detected by the six-axissensor 111 in a state where the track pad 14 receives an operation.Thereby, it is possible to prevent a user's unintended change in displayby the operation state of the track pad 14.

In addition, the control unit 150 performs the first display change in acase where the movement in the first movement direction is detected bythe six-axis sensor 111 in a state where the track pad 14 receives anoperation. Thereby, it is possible to prevent a user's unintended changein display by the operation state of the track pad 14.

In addition, the control unit 150 starts the first display change in acase where the movement in the first movement direction is detected bythe six-axis sensor 111, and then stops the first display change in acase where the track pad 14 does not receive an operation. Thereby, itis possible to prevent a user's unintended change in display by theoperation state of the track pad 14.

In addition, the control unit 150 performs the second display change ina case where the movement in the second movement direction is detectedby the six-axis sensor 111 in a state where the track pad 14 receives anoperation. Thereby, it is possible to move an image displayed on theimage display unit 20 by moving the control device 10 and to indicate adirection of movement of the image in accordance with a movementdirection of the control device 10.

In addition, the operation reception unit including the track pad 14receives an operation which is different from detection performed by thesix-axis sensor 111 as a movement detection unit, or the like. For thisreason, it is possible to select a state in which display is controlledand a state where display is not controlled, in accordance with anoperation which is different from the movement of the control device 10which is detected by the six-axis sensor or the like. For this reason,it is possible to more reliably prevent a user's unintended change indisplay.

Here, an operation of the track pad 14 is described as a specificexample of an operation which is different from detection performed bythe six-axis sensor 111 as a movement detection unit in theabove-described embodiment. That is, a configuration in which anoperation detection unit is equivalent to the track pad 14 is described.However, the invention is not limited thereto, and the up and down keys15, the changeover switch 16, the power supply switch 18, or the likemay detect an operation which is different from detection performed bythe movement detection unit, as an operation detection unit. In a casewhere the six-axis sensor 111 which is a movement detection unit detectsan operation of moving the case 10A, an operation of knocking on thecase 10A may be set as the above-mentioned “different operation”. Inthis case, the control device 10 includes a knock sensor and a pressuresensor, and detections of the sensors may be set as operations differentfrom detection performed by the six-axis sensor 111 as a movementdetection unit.

In the above-described embodiment, the control device 10 including thecase 10A has been described as an example of the operation deviceaccording to the invention, but the invention is not limited thereto.For example, the operation device may be an operation device, includinga sensor equivalent to a movement detection unit and a functional unitequivalent to an operation reception unit, which is a card typeoperation device that does not include a housing. In addition, theinvention can also be applied to an operation device which is formedintegrally with another device. In addition, the operation device canalso be referred to as, for example, a remote controller, a controldevice, a control device, a small-sized device, a motion device, or thelike.

In the above-described embodiment, a description has been given of aconfiguration in which the operation device is connected to the imagedisplay unit 20 in a wired manner, but the invention is not limitedthereto. The operation device and the image display unit 20 may beconfigured to be connected to each other in a wireless manner. In thiscase, a method described as a communication method corresponding to thecommunication unit 117 may be adopted as a wireless communicationmethod, or another communication method may be used.

In addition, the operation device is not required to have the functionof the control device 10. Apart from the control device 10, anapparatus, a device, a unit, or an instrument which is used as anoperation device may be provided.

For example, the invention can also be applied by using a wearabledevice, capable of being attached to a user's body, clothes, or anaccessory worn by a user as an operation device, instead of the controldevice 10. In this case, the wearable device may be, for example, atimepiece type device, a ring type device, a laser pointer, a mouse, anair mouse, a game controller, a pen type device, or the like.

Here, another example of the operation device will be described.

FIG. 13 is a diagram showing the exterior of a timepiece type device 3.The timepiece type device 3 can be used as another example of theoperation device. The timepiece type device 3 is used together with, forexample, the control device 10. In this case, the control device 10 maybe configured not to function as an operation device.

The timepiece type device 3 includes a band portion 300 having the sameshape as a band of a wristwatch. The band portion 300 includes afixation portion, such as a buckle, which is not shown in the drawing,and can be fixed by being wound around, for example, a user's forearmportion. A substantially disk-shaped plane portion 300A is formed at aposition equivalent to a dial of a timepiece in the band portion 300 ofthe timepiece type device 3. The plane portion 300A is provided with abezel 301, an LCD 303, a button 305, a winding crown type operator 307,and a plurality of buttons 309.

The bezel 301 is a ring-shaped operator, and is disposed at a peripheraledge of the plane portion 300A. The bezel 301 is provided so as to berotatable with respect to the band portion 300 in the circumferentialdirection. The timepiece type device 3 includes a mechanism that detectsa rotation direction and the amount of rotation of the bezel 301 asdescribed later. In addition, the mechanism that rotatably supports thebezel 301 against the band portion 300 may include a storage thatgenerates a notch sound in accordance with rotation.

The LCD 303 is a liquid crystal display that displays characters or animage.

The button 305 is a push button type switch which is disposed on theouter side of the bezel 301. The button 305 is positioned below thebezel 301 when seen from a user, in a wearing state of the timepiecetype device 3. The button 305 has a size larger than those of thewinding crown type operator 307 and the button 309, and can also beoperated by touch.

The winding crown type operator 307 is an operator having a dragonheadshape of a wristwatch, and can be rotated as indicated by an arrow inthe drawing. The timepiece type device 3 includes a mechanism thatdetects a rotation direction and the amount of rotation of the windingcrown type operator 307 in a case where a user rotates the winding crowntype operator 307. In addition, a mechanism that rotatably supports thewinding crown type operator 307 against the band portion 300 may includea storage that generates a notch sound in accordance with rotation.

The button 309 is a push button type switch which is provided at theouter peripheral portion of the plane portion 300A. The number ofbuttons 309 is not particularly limited, and an example in which fourbuttons 309 are provided is described in the present embodiment.

Different functions can be allocated to the buttons 309, and thefunctions allocated to the respective buttons 309 can be displayed onthe LCD 303.

The timepiece type device 3 can use the bezel 301, the button 305, thewinding crown type operator 307, and the buttons 309 as operationreception units. These operation reception units are allocated to afunction of giving an instruction for the execution and/or cancellationof a mute function of temporarily stopping the display of the imagedisplay unit 20 or a function of giving an instruction for the change ofa type of pointer displayed on the image display unit 20, for example,during a normal operation.

In addition, the timepiece type device 3 has a communication unit and amotion sensor built therein. The communication unit, not shown in thedrawing, functions similarly to the communication unit 117, and themotion sensor, not shown in the drawing, functions similarly to thesix-axis sensor 111. The timepiece type device 3 detects an operationwith respect to each unit functioning as an operation reception unit,and transmits a detection result to the control device 10 in a wirelessmanner. In addition, the timepiece type device 3 transmits a detectionresult of the motion sensor to the control device 10 in a wirelessmanner.

In such a configuration, the control device 10 can detect an operationof moving the timepiece type device 3 like, for example, theabove-mentioned motions A to F, C′, D′, E′, and F′. In addition, thecontrol device 10 detects operations of the bezel 301, the button 305,the winding crown type operator 307, and the buttons 309 as operationsdifferent from detection performed by the motion sensor. In this case,it is possible to control the display of the image display unit 20 inaccordance with an operation of moving the timepiece type device 3 whileoperating the bezel 301, the button 305, the winding crown type operator307, and the buttons 309. A movement direction of the timepiece typedevice 3 and the control of display of the image display unit 20 can beperformed, for example, in the same manner as those in theabove-described embodiment.

FIG. 14 is a diagram showing the exterior of a ring type device 4. Thering type device 4 has the same shape as a ring, and is put on, forexample, a user's finger. A track pad 401 capable of detecting a contactoperation, similar to the track pad 14, is installed on the surface ofthe ring type device 4. The ring type device 4 detects a contact on thetrack pad 401 by a detection circuit not shown in the drawing. In such aconfiguration, the track pad 401 functions as an operation detectionunit. The track pad 401 is allocated to a function of giving aninstruction for the execution and/or cancellation of a mute function oftemporarily stopping the display of the image display unit 20 or afunction of giving an instruction for the change of a type of pointerdisplayed on the image display unit 20, for example, during a normaloperation.

In addition, the ring type device 4 has a communication unit and amotion sensor built therein. The communication unit, not shown in thedrawing, functions similarly to the communication unit 117, and themotion sensor, not shown in the drawing, functions similarly to thesix-axis sensor 111. The ring type device 4 detects an operation withrespect to each unit functioning as an operation reception unit, andtransmits a detection result to the control device 10 in a wirelessmanner. In addition, the ring type device 4 transmits a detection resultof the motion sensor to the control device 10 in a wireless manner.

In such a configuration, the control device 10 can detect an operationof moving the ring type device 4 like, for example, the above-mentionedmotions A to F, C′, D′, E′, and F′. In addition, the control device 10detects an operation of the track pad 401 as an operation which isdifferent from detection performed by the motion sensor. In this case,it is possible to control the display of the image display unit 20 inaccordance with an operation of moving the ring type device 4 whileoperating the track pad 401. A movement direction of the ring typedevice 4 and the control of display of the image display unit 20 can beperformed, for example, in the same manner as those in theabove-described embodiment.

In a case where a wearable device worn on a user's body, like thetimepiece type device 3 and the ring type device 4, is used as anoperation device, it is possible to easily control display in accordancewith an operation of moving the user's body. In addition, the control ofdisplay based on a detected value of the motion sensor is performed bydetecting an operation different from detection performed by the motionsensor, and thus it is possible to prevent a user's unintended change indisplay. For this reason, an instinctive operation is realized, and itis possible to prevent an erroneous operation and to control a user'sintended display.

A direction of movement of the timepiece type device 3 and the ring typedevice 4 can be detected and specified on the basis of, for example, theX-axis direction, the Y-axis direction, and the Z-axis direction shownin the drawing. A motion of moving the timepiece type device 3 and thering type device 4 in a specific direction is equivalent to an operationin a first movement direction and an operation in a second movementdirection, similar to the above-mentioned motions A to F, C′, D′, E′,and F′. In addition, the operation in the first movement direction andthe operation in the second movement direction are set as movements in aspecific direction, but a movement close to the specific direction maybe regarded as a movement in the specific direction. A range in whichmotions are regarded as motions in the same direction may be arbitrarilyset, and may be set to, for example, ±5 degrees or ±10 degrees. The sameis true of the first movement direction and the second movementdirection. In addition, a standard of a direction of each of themovement of the timepiece type device 3 and the ring type device 4 maybe set in advance in association with a detection direction of themotion sensor which is built into each of the timepiece type device 3and the ring type device 4. Therefore, the direction of movement can beset on the basis of any direction, and is not limited to the X-axisdirection, the Y-axis direction, and the Z-axis direction shown in thedrawing.

In addition, the timepiece type device 3 and the ring type device 4 arenot limited to being used together with the control device 10. The HMD100 may have a function equivalent to the control device 10 and may beconfigured to be capable of controlling the display of the image displayunit 20. Accordingly, when a configuration in which units including themain processor 140 can be accommodated in the image display unit 20 isadopted, the HMD 100 can be constituted by, for example, the imagedisplay unit 20 and the timepiece type device 3 or the image displayunit 20 and the ring type device 4.

Meanwhile, the invention is not limited to the configuration of theabove-described embodiment, and can be implemented in various aspectswithout departing from the scope of the invention.

For example, in the above-described embodiment, a configuration in whicha user visually perceives an outside scene through a display unit is notlimited to a configuration in which the right light guiding plate 26 andthe left light guiding plate 28 transmit external light. For example,the invention can also be applied to a display device capable ofdisplaying an image in a state where an outside scene cannot be visuallyperceived. Specifically, the invention can be applied to a displaydevice that displays an image captured by the camera 61, an image or CGwhich is generated on the basis of the captured image, a video based onvideo data stored in advance or video data which is input from theoutside, or the like. Such types of display devices may include aso-called closed display device by which an outside scene cannot bevisually perceived. In addition, the invention can also be applied to adisplay device that does not perform processing such as AR display inwhich an image is displayed so as to overlap a real space as describedin the above-described embodiment, mixed reality (MR) display in which acaptured image of a real space and a virtual image are combined witheach other, or virtual reality (VR) display in which a virtual image isdisplayed. For example, a display device that displays video data or ananalog video signal which is input from the outside can also be, ofcourse, included in objects to which the invention is applied.

In the above-described embodiment, enlargement, reduction, scrolling,and changeover of an image based on the display order have beendescribed as examples in which a display mode of an image displayed onthe image display unit 20 is changed, but the invention is not limitedthereto. For example, control of rotating an image, control of changingdisplay luminance of an image, control of increasing or reducing thedisplay size of a portion of an image being displayed or characters, andthe like may be performed in response to a motion. In this case, arotation direction of the image, a change direction (an increase ordecrease in luminance) of the display luminance of the image, and theincrease or reduction in size may be associated with different motions.In addition, the operation of a trigger for starting an operation basedon a motion is not limited to being performed using the track pad 14 oran operator such as the button included in the control device 10, andmay be detected by a method different from the movement of the case 10A.Specifically, since the operation may be able to be detected based on adetected value different from a detected value of the six-axis sensor111, for example, the movement of the image display unit 20 which isdetected by the six-axis sensor 235 may be set as a trigger operation.In addition, for example, a touch sensor (not shown) is provided in theimage display unit 20, and an operation with respect to the touch sensormay be set as a trigger operation.

In addition, for example, another type of image display unit such as animage display unit worn like, for example, a cap may be adopted insteadof the image display unit 20, and the image display unit may include adisplay unit displaying an image corresponding to a user's left eye anda display unit displaying an image corresponding to the user's righteye. In addition, the display device according to the invention may beconfigured as a head-mounted display which is mounted on a vehicle suchas an automobile or an airplane. In addition, the display device may beconfigured as a head-mounted display which is built into a bodyprotection tool such as a helmet. In this case, a portion positioning aposition with respect to a user's body and a portion positioned withrespect to the portion can be set to be a mounting portion.

Further, in the above-described embodiment, a description has been givenof an example of a configuration in which the image display unit 20 andthe control device 10 are separated from each other and are connected toeach other through the connection cable 40, but the control device 10and the image display unit 20 can be integrally formed and can also beconfigured to be worn on a user's head.

In addition, a notebook computer, a tablet computer, or a desktopcomputer may be used as the control device 10. In addition, a portableelectronic device including a game machine, a mobile phone, a smartphone, and a portable media player, other dedicated devices, and thelike are used as the control device 10. In addition, a configuration mayalso be adopted in which the control device 10 is configured so as to beseparated from the image display unit 20, and various signals aretransmitted and received between the control device 10 and the imagedisplay unit 20 through wireless communication.

In the above-described embodiment, a description has been given of anexample of a configuration in which the image display unit 20 and thecontrol device 10 are separated from each other and are connected toeach other through the connection cable 40, but a configuration may alsobe adopted in which the control device 10 and the image display unit 20are connected to each other through a wireless communication line.

In addition, a description has been given of a configuration in which avirtual image is formed in a portion of each of the right light guidingplate 26 and the left light guiding plate 28 by the half mirrors 261 and281, as an optical system that guides image light to a user's eyes. Theinvention is not limited thereto, and a configuration may also beadopted in which an image is displayed in a display region having anarea that occupies the entirety or majority of the right light guidingplate 26 and the left light guiding plate 28. In this case, in anoperation of changing a display position of an image, a process ofreducing the size of the image may be included.

Further, the optical element according to the invention is not limitedto the right light guiding plate 26 and the left light guiding plate 28which respectively include the half mirrors 261 and 281, and may be anoptical component that makes image light incident on a user's eyes.Specifically, a diffraction grating, a prism, or a holographic displayunit may be used.

In addition, at least a portion of functional blocks shown in FIGS. 4and 5 may be realized by hardware or may be realized by cooperation ofhardware and software, and the invention is not limited to aconfiguration in which an independent hardware resource is disposed asshown in the drawing. In addition, a program executed by the controlunit 150 may be stored in another storage device (not shown) within thenon-volatile storage unit 121 or the control device 10. In addition, aconfiguration may also be adopted in which a program stored in anexternal device is acquired and executed through the communication unit117 and the external connector 184. In addition, among the componentsformed in the control device 10, the operation unit 110 may be formed asa user interface (UI). In addition, the components formed in the controldevice 10 may also be formed in the image display unit 20. For example,the same processor as the main processor 140 may be disposed in theimage display unit 20, and the main processor 140 included in thecontrol device 10 and the processor of the image display unit 20 may beconfigured to perform separate functions.

The entire disclosure of Japanese Patent Application Nos. 2016-026989,filed Feb. 16, 2016 and 2016-026990, filed Feb. 16, 2016 are expresslyincorporated by reference herein.

What is claimed is:
 1. A head-mounted display device including a displayunit that makes a user visually perceive an image, the head-mounteddisplay device comprising: an operation device, capable of being movedindependently of the display unit, which includes a movement detectionunit detecting a movement of the operation device and an operationreception unit receiving an operation different from the detection ofthe movement detection unit; and a control unit that controls display ofthe display unit in accordance with the movement detected by themovement detection unit in a state where the operation reception unitreceives an operation.
 2. The display device according to claim 1,wherein the operation reception unit receives at least any one of apressing operation and a contact operation.
 3. The display deviceaccording to claim 1, wherein the control unit changes a displaymagnification of an image displayed on the display unit in a case wherea movement corresponding to an operation of rotating the operationdevice is detected by the movement detection unit.
 4. The display deviceaccording to claim 1, wherein the control unit changes an imagedisplayed on the display unit in a direction which is set in advance, ina case where a movement corresponding to an operation of lifting downthe operation device or an operation of lifting up the operation deviceis detected by the movement detection unit.
 5. The display deviceaccording to claim 1, wherein the control unit changes display of thedisplay unit in a state where the operation reception unit receives anoperation, and then fixedly sets a change in the display of the displayunit in a case where the operation is canceled.
 6. The display deviceaccording to claim 1, wherein the control unit has a predeterminedfunction set therein which is performed in a case where the operationreception unit receives an operation, and performs a normal mode inwhich the predetermined function is performed in a case where theoperation reception unit receives an operation and a display operationmode in which the display of the display unit is controlled inaccordance with the movement detected by the movement detection unit ina state where the operation reception unit receives an operation, in achangeover manner.
 7. The display device according to claim 6, whereinthe control unit performs the display operation mode in a case where theoperation reception unit receives an operation within a set time afterthe movement detection unit detects start of a movement.
 8. The displaydevice according to claim 7, wherein in a case where the control unitstarts performing the display operation mode, the control unit reflectsa detection history detected by the movement detection unit during aperiod between when the movement detection unit detects start of amovement and when the operation reception unit receives an operation, tothereby control display of the display unit.
 9. The display deviceaccording to claim 1, wherein the operation device includes a positionindication operation unit that receives a position indication operation.10. The display device according to claim 9, wherein the control unitchanges display of the display unit on the basis of a display positioncorresponding to a position which is indicated by a position indicationoperation received by the position indication operation unit in a statewhere the position indication operation is received by the positionindication operation unit and the operation reception unit receives anoperation.
 11. The display device according to claim 10, wherein thecontrol unit displays an image displayed on the display unit in anenlarged or reduced manner, centering on the display positioncorresponding to the position which is indicated by the positionindication operation received by the position indication operation unit.12. The display device according to claim 9, wherein the positionindication operation unit functions as the operation reception unit. 13.The display device according to claim 10, further comprising: a displayunit movement detection unit that detects a movement of the displayunit, wherein the control unit specifies an indication position of anoperation received by the position indication operation unit at a timingwhen the display unit movement detection unit detects a predeterminedmovement.
 14. The display device according to claim 1, wherein thecontrol unit performs control of moving an image displayed on thedisplay unit in a first display operation direction and a second displayoperation direction paired with the first display operation direction,and wherein the first display operation direction is associated with afirst movement direction detected by the movement detection unit, andthe second display operation direction is associated with a secondmovement direction detected by the movement detection unit.
 15. Thedisplay device according to claim 1, wherein the operation device is awearable device which is worn on the user's body.
 16. A head-mounteddisplay device including a display unit that makes a user visuallyperceive an image, the head-mounted display device comprising: anoperation device, capable of being moved independently of the displayunit, which includes a movement detection unit detecting a movement ofthe operation device; and a control unit that controls display of thedisplay unit in accordance with the movement detected by the movementdetection unit, wherein the control unit performs a first display changecorresponding to a movement in a first movement direction which isdetected by the movement detection unit and a second display changecorresponding to a movement in a second movement direction which isdetected by the movement detection unit, as a process performed on animage displayed on the display unit, and wherein the control unit startsthe first display change in a case where the movement in the firstmovement direction is detected, and then stops the first display changein a case where the movement in the second movement direction isdetected by the movement detection unit.
 17. The display deviceaccording to claim 16, wherein the control unit performs the firstdisplay change in a case where the movement in the first movementdirection is detected by the movement detection unit, and performs thesecond display change in a case where the movement in the secondmovement direction is detected by the movement detection unit.
 18. Thedisplay device according to claim 16, wherein the operation deviceincludes an operation reception unit that receives an operation, andwherein the control unit changes display of the display unit inaccordance with the movement detected by the movement detection unit ina state where the operation reception unit receives an operation. 19.The display device according to claim 16, wherein the operation deviceis a wearable device which is worn on the user's body.
 20. A method ofcontrolling a display device including a display unit that makes a uservisually perceive an image and an operation device, capable of beingmoved independently of the display unit, which includes a movementdetection unit detecting a movement of the operation device and anoperation reception unit receiving an operation, the method comprising:controlling display of the display unit in accordance with the movementdetected by the movement detection unit in a state where the operationreception unit receives an operation.
 21. A method of controlling adisplay device including a display unit that makes a user visuallyperceive an image and an operation device, capable of being movedindependently of the display unit, which includes a movement detectionunit detecting a movement of the operation device, the methodcomprising: performing a first display change corresponding to amovement in a first movement direction which is detected by the movementdetection unit and a second display change corresponding to a movementin a second movement direction which is detected by the movementdetection unit, as a process performed on an image displayed on thedisplay unit; and starting the first display change in a case where themovement in the first movement direction is detected, and then stoppingthe first display change in a case where the movement in the secondmovement direction is detected by the movement detection unit.